Optimod – An automated approach for constructing and optimizing initial models for single-particle electron microscopy

Abstract

Single-particle cryo-electron microscopy is now well established as a technique for the structural characterization of large macromolecules and macromolecular complexes. The raw data is very noisy and consists of two-dimensional projections, from which the 3D biological object must be reconstructed. The 3D object depends upon knowledge of proper angular orientations assigned to the 2D projection images. Numerous algorithms have been developed for determining relative angular orientations between 2D images, but the transition from 2D to 3D remains challenging and can result in erroneous and conflicting results. Here we describe a general, automated procedure, called OptiMod, for reconstructing and optimizing 3D models using common-lines methodologies. OptiMod approximates orientation angles and reconstructs independent maps from 2D class averages. It then iterates the procedure, while considering each map as a raw solution that needs to be compared with other possible outcomes. We incorporate procedures for 3D alignment, clustering, and refinement to optimize each map, as well as standard scoring metrics to facilitate the selection of the optimal model. We also show that small angle tilt-pair data can be included as one of the scoring metrics to improve the selection of the optimal initial model, and also to provide a validation check. The overall approach is demonstrated using two experimental cryo-EM data sets – the 80S ribosome that represents a relatively straightforward case for ab initio reconstruction, and the Tf–TfR complex that represents a challenging case in that it has previously been shown to provide multiple equally plausible solutions to the initial model problem.

1. Introduction

Single-particle analysis using cryo-electron microscopy (cryo-EM) is now a well established tool in structural biology and is enabling the 3D reconstruction of large macromolecular complexes at resolutions ranging from nanometer (Lander et al., 2012) to near-atomic (Bai et al., 2013; Campbell et al., 2012; Li et al., 2013). Unlike X-ray crystallographic methods, the technique is not constrained by the requirements of crystallization and provides the capability to examine samples in their near native state embedded within a layer of vitreous ice (Adrian et al., 1984). Ideally the macromolecular objects, or “single particles”, will adopt numerous orientations within the ice layer, and transmission electron microscopy provides images that are 2D projections of these 3D objects. The orientation of each particle relative to any other is unknown, but can be described by 2 translations (x and y) and 3 rotational Euler angles (φ, θ, and ϕ). These five parameters must be determined in order to reconstruct a 3D map of the object. Due to multiple sources of noise during imaging, each particle image has a very low signal-to-noise ratio; typically ~0.05 for large objects (>1000 kDa) such as the ribosome (Baxter et al., 2009), but potentially much lower for smaller objects or if the imaging conditions are not ideal.

The combined effect of unknown object orientations and low signal-to-noise ratio can make the transition from noisy 2D projection images to a reliable 3D reconstruction challenging. While this problem is computationally hard (Mielikäinen et al., 2004), the task is manageable when an initial approximation exists for the overall shape of the 3D object – i.e. its molecular envelope. Indeed, many refinement packages (Frank et al., 1996; Grigorieff, 2007; Heymann and Belnap, 2007; Hohn et al., 2007; Scheres, 2012; Sorzano et al., 2004; Tang et al., 2007; van Heel et al., 1996) can accurately address this problem when an initial model is available. However, the task is particularly complicated in the absence of prior 3D information. This is because, in practice, unknown orientation angles and low signal-to-noise ratio can often lead to local maxima during the orientation search and provide multiple solutions to the problem. Thus, the task of constructing an initial model has received much attention (Bartesaghi et al., 2012; Cheng et al., 2006; Elmlund and Elmlund, 2012; Radermacher et al., 1986; van Heel, 1987; Voss et al., 2010). The problem can be successfully addressed using a variety of methods (Voss et al., 2010), some of which require physical specimen tilting inside the microscope, but each of which has particular advantages and disadvantages. Ideally, one would collect a single data set that is intended to achieve the highest possible resolution and derive ab initio orientations from the images themselves, leaving some time during data collection for the acquisition of a limited number of low-angle tilt-pairs, which would be used to validate the initial model and determine the absolute hand (Henderson et al., 2011; Rosenthal and Henderson, 2003). Such an approach would avoid the drawbacks associated with using tilted methods for initial model calculation, such as specimen flattening in negative stain, beam-induced motion at high tilt angles in cryoEM, or low resolution in tomography (Voss et al., 2010), and in general would avoid the necessity for collecting additional data.

Constructing an initial 3D model without resorting to tilted methods is still considered a relatively challenging task, and one that is prone to error (Cheng et al., 2006; Voss et al., 2010). For certain samples, in particular homogeneous icosahedral viruses, preliminary structure determination can be very robust and reproducible using a random initial model (essentially a Gaussian sphere) and a procedure for iteratively refining that model to convergence (Yan et al., 2007). This is in part due to the fact that the high (60-fold) symmetry limits the number of possible orientations that the algorithm must search. For macromolecules with lower symmetry, and especially for asymmetric structures, more sophisticated procedures for performing automated angular assignment are required. The majority of such algorithms are based on the central section theorem, all of which attempt to identify angular relationships by searching for pairs of matching 1D lines that are by definition shared between any two 2D projections arising from an identical 3D object (Crowther et al., 1970; Elmlund and Elmlund, 2012; Elmlund et al., 2009; Goncharov and Vainshtein, 1986; Penczek et al., 1996; Singer et al., 2010; van Heel, 1987). These methods have enabled the characterization of a number of complex macromolecules (Elmlund et al., 2010; Elmlund and Elmlund, 2009; Serysheva et al., 1995). Unfortunately, in the absence of additional tilted data, such approaches have sometimes also produced conflicting results of complexes that might be expected to be similar or identical (da Fonseca et al., 2003; Hamada et al., 2003; Sato et al., 2004; Serysheva et al., 2003; Thrower et al., 2002).

To facilitate and optimize the determination of initial models in single-particle EM, we have developed a procedure that we call OptiMod. The method incorporates multiple automated algorithms for determining orientations using common-lines methodologies, and provides criteria for scoring the results. Rather than constructing a single 3D map from a common-lines based reconstruction routine, OptiMod generates multiple maps using algorithm-specific randomizations, but treats each result as a raw solution, one that needs to be compared to all other possible outcomes, and subsequently optimized and validated. To achieve this outcome, OptiMod aligns and classifies the raw 3D maps, refines the data set using each of the 3D classes as a unique initial model, and then analyzes each refined map using a standard scoring metric. Any scoring metric can be incorporated into the method, provided that the metric itself can accurately discriminate a correct from an incorrect result, which we demonstrate here using a small-angle tilt test (Henderson et al., 2011; Rosenthal and Henderson, 2003). The routines are relatively inexpensive in terms of computation time, so that a reliable map can be generated in a few hours on a single multi-core machine. Here we describe the approach and present two experimental test cases to which the method has been successfully applied. The method has been integrated into Appion (Lander et al., 2009) and is also is available as a standalone application (available at http://nramm.scripps.edu).

2. Algorithm description

2.1. Overall aim of OptiMod

The overall aim of OptiMod is to determine an optimal initial model for high-resolution single-particle based refinement. More specifically, OptiMod aims to: (1) automate the construction of many initial models using subsets of the data, (2) assess the ability of the data to converge upon a single reproducible 3D structure by analyzing the resulting initial models using one or several scoring criteria that are standard in the field, and (3) select the optimal initial model. The only input to the procedure consists of a set of class averages, while the output is a set of initial models, optimized for internal consistency with the input data and ordered according to an overall quality score.

2.2. Organization of OptiMod

OptiMod is subdivided into six distinct procedures (Fig. 1): (0) class average pre-processing to align and scale the input 2D images, (1) iterative, raw volume reconstruction using a common-lines based strategy, (2) raw volume 3D alignment, (3) aligned 3D volume clustering and averaging within homogeneous groups, (4) averaged volume refinement, and (5) refined volume assessment using one or multiple scoring metrics. These individual procedures are combined into a single routine, and their details are described below.

Fig. 1.

Schematic of OptiMod: OptiMod is divided into 6 procedures – numbered and shaded regions represent specific algorithmic methods. (0) The first step is pre-processing of the class averages and is optional; options are provided to iteratively align, center, and/or scale the 2D class averages. (1) Multiple raw volumes are iteratively constructed using a common-lines based approach for Euler-angle assignment (see also Supplementary Fig. 1). (2) All raw volumes are aligned to a common scaffold. (3) Pair-wise similarities are calculated between all aligned volumes, enabling clustering into homogeneous groups to produce clustered and averaged volumes. (4) Each clustered and averaged volume is refined against a set of class averages (either from the original input, or separately specified by the user). (5) The refined volumes are assessed using one or several scoring metrics, and the best is selected.

2.3. OptiMod procedures

All processing was performed on a single node of a Dell Power-edge 1955 blade server running CentOS6 linux with 8 cores and 16 Gb of memory. The approximate computation time for each procedure is indicated at the end of the section.

• (0)Pre-processing (Fig. 1, procedure 0): options are provided to rotationally and translationally align the class averages to each other, to center the class averages, and to scale the class averages prior to iterative raw volume reconstruction (Table 1). Typically, either an alignment or a centering operation is sufficient (not both). The scaling parameter reduces the class averages to a 64 × 64 box size for efficiency, and in test runs has always been beneficial. Procedure 0 takes ~1 min for 50 class averages.
• (1)Iterative raw volume reconstruction (Fig. 1, procedure 1): a set of class averages is used as input. For optimal results, the input classes should represent as many views of the imaged object as are present in the data, and these views should be approximately equally represented in the input. Orientation angles are assigned to the distinct classes. Currently, this is accomplished either using the Angular Reconstitution procedure (van Heel, 1987) or the cross-common lines procedure implemented in EMAN1 (Ludtke et al., 1999), but any common-lines based methodology is suitable and can be readily incorporated. A protocol designed to reduce bias, while simulating the decision-making process of an experienced user of the image processing package is incorporated into the reconstruction procedure. In the case of angular reconstitution (Supplementary Fig. 1), this is intended to provide orthogonal views of the imaged objects as initial input. OptiMod first calculates pair-wise similarities (and by analogy, dissimilarities) between input class averages. Subsequently, it weights the randomized sequence of image addition into angular reconstitution in accordance with the highest cumulative dissimilarity. In the case of the cross-common lines methodology, OptiMod simply selects a random subset of images with which to calculate the initial model (usually ~50%), since all input class averages contribute equally to the common-lines search. The process of Euler angle determination and 3D reconstruction for a single model is iterated multiple times to produce n raw 3D volumes (this number, n, is specified by the user [see also Table 1]). Typically, one hundred to one thousand iterations are performed. Procedure 1 takes ~18 h to calculate 1000 raw volumes from 50 64 × 64 pixel class averages.
• (2)Raw volume 3D alignment (Fig. 1, procedure 2): the goal of any alignment and clustering, whether in 2D or 3D, is to identify the predominant signal in the data by extracting and bringing into register subsets of similar objects within the data. OptiMod uses the same method that has been established for tomography, wherein each tomogram represents a raw and noisy preliminary reconstruction and must be aligned to its peers prior to classification in order to increase the signal-to-noise ratio of the raw maps. In OptiMod, each raw volume calculated in procedure 1 is aligned using a 3D maximum-likelihood (ML) alignment (Scheres et al., 2009). Multiple classes can be requested in the ML approach and used as references for 3D alignment (and effectively classification). This will, in principle, provide a better registration by accounting for different shapes and sizes in the starting pool of raw volumes. The multiple 3D ML classes can also serve as starting points for 3D refinement (procedure 4), skipping procedure 3 altogether (described below). The only practical downside to using ML with more than a single reference and skipping the default method for 3D clustering is computation cost, which would scale linearly as the number of ML classes is increased. Procedure 2 takes ~6 h to align 1000 64 × 64 × 64 pixel raw volumes.
• (3)Aligned volume 3D clustering (Fig. 1, procedure 3): raw 3D volumes produced and aligned in the previous procedure may be inconsistent with one another and with the input class averages (Cheng et al., 2006; Voss et al., 2010). Noise contributions resulting from improperly assigned orientations, compositional or conformational heterogeneity, or translational misalignment are of significant concern and may adversely affect the resulting map. Aligned 3D volumes are thus clustered into homogeneous groups using the affinity propagation strategy (Frey and Dueck, 2007) to increase the signal-to-noise ratio. The similarity between any two aligned 3D volumes is determined either through direct 3D cross-correlation or through the inverse of the Euclidean distance between hyperspace coordinates after dimensionality reduction by principal components analysis. The latter approach is analogous to the 2D case (van Heel and Frank, 1981) and provides significant improvements in computational efficiency, while focusing on the primary variations within the data. Thus, 3D clusters are obtained (the number of clusters is governed by the similarities between raw volumes and is dataset dependent) and the volumes within them are averaged. Procedure 3 takes ~5 min to cluster 1000 64 × 64 × 64 pixel 3D volumes, provided that the dimensionality of the data is first reduced using PCA.
• (4)Averaged volume refinement (Fig. 1, procedure 4): aligned and averaged 3D volumes increase the SNR of each 3D map, and in many cases suffice as starting points for high-resolution refinement. In OptiMod, each clustered 3D volume is automatically refined against an input set of class averages (which can be different from that used in procedure 1). This step is intended to extract the maximum amount of information from the input 2D data, and additionally allow each model to converge upon an optimal solution, given the model’s structural characteristics and the quality of the class averages. Procedure 4 takes ~8 h to refine ~100 64 × 64 × 64-pixel clustered volumes (this number is dataset dependent and is produced from the raw volumes) using 50 class averages.
• (5)
  Refined volume assessment (Fig. 1, procedure 5): for each refined final electron density map, “goodness of fit” statistics are calculated, which are meant to assess the quality of the map using standard scoring metrics commonly employed in single-particle EM. A standard similarity measurement is the correlation coefficient between two images, X and Y, each with pixel values i:

  $$\mathit{CCC}(X,Y)=\frac{{\displaystyle \sum _{i=1}^i}\left(X_i-\overline{X}\right)\left(Y_i-\overline{Y}\right)}{\sqrt{{\displaystyle \sum _{i=1}^i}{\left(X_i-\overline{X}\right)}^2{\displaystyle \sum _{i=1}^i}{\left(Y_i-\overline{Y}\right)}^2}}$$ (1)
  In OptiMod, image X is the projection (class average) and image Y is the re-projection of the averaged and refined 3D volume along the identified Euler angles. OptiMod calculates: (1) the value for the average cross-correlation coefficient between input projections, P, and re-projections, R, taken over the entire set of projection/re-projection pairs, j:

  $$\mathit{CCC}(P,R)=(1/j)\sum _{j=1}^j\left(\frac{{\displaystyle \sum _{i=1}^i}(P_i-\overline{P})(R_i-\overline{R})}{\sqrt{{\displaystyle \sum _{i=1}^i}{(P_i-\overline{P})}^2{\displaystyle \sum _{i=1}^i}{(R_i-\overline{R})}^2}}\right)$$ (2)

Table 1.

OptiMod processing parameters: The parameters described in this table have been observed to have an effect on the resulting initial models. For a detailed description of other OptiMod parameters, the reader is referred to the documentation that is distributed with the code.

OptiMod procedure	Parameter	Description & comment	Execution package
Pre-processing (0)	–prealign*	Iteratively align class averages to each other prior to common-lines Euler search (performs a translational and rotational alignment). This parameter can be very helpful in cases when the class averages may not be translationally aligned, which would produce poor results during the Euler search	IMAGIC
Pre-processing (0)	–center*	Center class averages prior to common-lines Euler search. This parameter can be very helpful in cases when the class averages may not be translationally aligned, which would produce poor results during the Euler search	EMAN
Pre-processing (0)	–scale	Scale class averages to 64 × 64 pixels prior to Euler search. This parameter can dramatically speed up processing and is usually recommended	EMAN
Iterative raw volume reconstruction (1)	–num_volumes	Number of raw volumes to generate. The recommended value is between 100 and 1000	IMAGIC/EMAN
Iterative raw volume reconstruction (1)	–threes	Rather than using all of the class averages for the raw volume calculations, if this option is specified, all raw volumes will be built only from 3 images, as per the original angular reconstitution theory. This parameter has been mostly useful to speed up Euler search and raw volume generation	Python/IMAGIC
Iterative raw volume reconstruction (1)	–images_per_volume	If using EMAN cross-common lines, this parameter refers to the number of images to use for constructing each 3D map. The input averages correspond to the total ‘pool’ from which a subset will be selected for analysis by cross-common lines. The recommended default is to specify a value that is 50% of the total number of input class averages	EMAN
3D alignment (2)	–nref	Describes the number of 3D alignment references to be generated using the maximum-likelihood approach. It is theoretically possible to improve the 3D alignment by using more references, albeit at the expense of computation cost	Xmipp
3D refinement (4)	–refine_classavgs	Points to a distinct stack of class averages to use for the refinement of clustered models. It can be very beneficial to provide a large number of class averages here in order to help refine the poorer clustered 3D models	Xmipp

^*Parameter has been observed to have an influence on the resulting initial models.

This value will be referred to as CCC_PR from now on. OptiMod also calculates (2) the Fourier shell correlation between half sets of the class averages, which measures the resolution of the averaged and refined 3D volume (FSC) (Harauz and van Heel, 1986). Optimod uses the CCC_PR metric to determine the best model. In principle, any number of scoring metrics can be incorporated and combined into a single criterion, as shown below for the small angle tilt test (see Section 4.3). The only requirement is that the scoring metric itself can accurately predict the correct result. Procedure 5 takes ~30 min to assess ~100 refined volumes that were clustered from the original 1000 starting volumes.

2.4. Important OptiMod parameters and initial model determination strategy

The parameters that have been found to affect the results of a typical OptiMod run are listed and described in Table 1. Most importantly, it is often useful to either align the class averages to each other or center the class averages prior to angular assignment, both of which can be performed internally inside OptiMod (Fig. 1, procedure 0). Typically, one would then run two OptiMod calculations – one where the class averages have been pre-processed using one of the two methods and one without this pre-processing (ideally, the two runs would otherwise be identical). The rest of the parameters listed in Table 1 have been found to have modest effects on the output and are dataset dependent. They can also be tested using a similar strategy. The suggested initial model determination strategy is to run 2 (or 4, if one wants to assess optional parameters) simultaneous OptiMod jobs, each, for example, with – num_volumes = 200: (1) default parameters; (2) default parameters + –prealign (or –center); (3) default parameters + [option]; (4) default parameters + –prealign (or –center) + [option].

3. Materials and methods

3.1. Electron microscopy specimen preparation and data collection for the 80S ribosome

A C-flat grid (Protochips, Inc.) with 2 μm holes was overlaid with 2 nm thin carbon and cleaned for 5 seconds using a Gatan Solarus plasma cleaner. 3 μL of yeast 80S ribosomes, at a concentration of 0.1 mg/mL, was applied to the grid, allowed to adsorb for 30 s, then plunged into liquid ethane using the FEI vitrobot after a 5 second blot. Data was acquired using the Leginon software (Suloway et al., 2005) installed on a Tecnai F20 Twin transmission electron microscope operating at 200 kV, with a dose of 20 e-/Ų and a nominal underfocus ranging from 1 to 4 μm. Images were collected and recorded at a nominal magnification of 80,000×, corresponding to a pixel size of 1.37 Å at the specimen level.

3.2. Image processing for the generation of class averages

3.2.1. 80S ribosome

All data was processed inside the Appion package unless otherwise stated (Lander et al., 2009). The CTF was estimated using the ACE2 package and corrected by applying a wiener-filter to the micrographs with a constant of 0.1. We used FindEM (Roseman, 2004) for particle selection. Single particles were extracted using a box-size of 320 pixels, and subsequently binned by 4 to provide a final box size of 80 pixels, corresponding to a pixel size of 5.48 Å at the specimen level. Preliminary unsupervised alignment and classification using the ML2D algorithm (Scheres et al., 2005) enabled the removal of visually bad particles, as described previously (Lyumkis et al., 2013b). The remaining stack was low-pass filtered to 20 Å and subjected to reference-based alignment using ten distinct centered references from ML2D above. The aligned and filtered stack was then used as input to the Iterative Stable Alignment and Clustering (ISAC) method for alignment and clustering (Yang et al., 2012). Unsupervised class averages for initial model calculation were obtained using ISAC (this step was performed outside of Appion).

3.2.2. Tf–TfR complex

A data set of 21,719 Tf–TfR particles collected previously (Cheng et al., 2004) was generously provided by Yifan Cheng. CTF correction was performed using the ACE2 package using the values previously identified by applying a wiener-filter to individual particles with a constant of 0.1. Unsupervised class averages from this set were calculated using the ISAC method for alignment and clustering (Yang et al., 2012) (this step was performed outside of Appion).

3.3. OptiMod model generation for class averages

3.3.1. 80S ribosome

50 reference-free class averages were selected as input to OptiMod (Fig. 2A). OptiMod was run using default parameters to automatically generate 112 clustered and refined models from 1000 raw volumes, which were ranked using the CCC_PR criterion. All electron density maps were displayed and rendered using UCSF Chimera (Pettersen et al., 2004).

Fig. 2.

Application of OptiMod – 80S ribosome: (A) Class averages used as input to OptiMod. (B) The scoring metric in OptiMod compared to the published map as measured by the Fourier shell correlation (0.5 threshold) between each output model and the EMDB structure. Data points for the CCC_PR metric (y-axis) used for selecting the optimal 3D reconstruction are plotted against the Fourier shell correlation between the model and the true solution (x-axis). Representative best, middle, and worst models, according to the final selection metric, are shown in green, orange, and red, respectively. The Pearson correlation coefficient (CC) is −0.82. (C) EMDB model of the 80S eukaryotic ribosome (EMD-1076). (D) 3D reconstructions from the result of procedure 3, and after refinement, procedure 4, are shown for the best, middle, and worst model selections indicated in (B). Scale bars are 250 Å.

3.3.2. Tf–TfR complex

From the full set of 180 calculated class averages (Supplementary Fig. 2) 56 were selected as input to OptiMod (Fig. 3A). OptiMod was run using default parameters to automatically generate 123 clustered and refined models from 1000 raw volumes, which were ranked using the CCC_PR criterion. For the exhaustive C1 refinement discussed in the results section, we performed 100 iterations of a standard projection-matching refinement protocol without constraining the Euler search (i.e. all orientations were globally sampled). For the symmetrized refinements, the symmetry axis was automatically determined using the align3dsym function implemented in EMAN1 (Ludtke et al., 1999). Each model output by OptiMod was subjected to automatic symmetry axis determination, aligned to the identified axis, and refined using 2-fold rotational symmetry. The results of symmetrized refinement are shown in Fig. 3D. All electron density maps were displayed and rendered using UCSF Chimera (Pettersen et al., 2004).

Fig. 3.

Application of OptiMod – Tf–TfR complex: (A) Class averages used as input to OptiMod. (B) The scoring metric in OptiMod compared to the published map as measured by the Fourier shell correlation (0.5 threshold) between each output model and the PDB structure that has been filtered to 30 Å resolution. Data points for the CCC_PR metric (y-axis) used for selecting the optimal 3D reconstruction are plotted against the Fourier shell correlation between the model and the true solution (x-axis). Representative best, middle, and worst models, according to the final selection metric, are shown in green, orange, and red, respectively. The Pearson correlation coefficient (CC) is −0.64. (C) PDB model of the Tf–TfR complex (PDB 1SUV), low-pass filtered to 30 Å. (D) 3D reconstructions from the result of procedure 3, and after refinement, procedure 4, are shown for the best, middle, and worst model selections indicated in (B), and also after automatic identification of the 2-fold symmetry-axis and refinement using C2 symmetry. Scale bars are 120 Å. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

3.4. Tilt-pair data acquisition and processing used in assessment of 3D ribosome models

Tilt-pair images were obtained with a tilt angle of 10° using the same grid and microscope as described above. Images were recorded at a nominal magnification of 50,000× using a TVIPS Tietz SCX 4 K × 4 K CCD camera, corresponding to a pixel size of 1.69 Å at the specimen level. An untilted and a tilted image were obtained of the same section of a grid, each with a dose of 16 e-/Ų, for a combined dose of 32 e-/Ų. This general strategy has been described previously (Henderson et al., 2011; Rosenthal and Henderson, 2003). 165 particle tilt-pairs were selected and binned 2× to give a pixel size of 3.38 Å. Initial orientations were determined for both stacks using each averaged and refined 3D model of the 80S ribosome (total 112 models) and a projection-matching routine in Xmipp. Subsequently, the orientations were refined with one cycle in Frealign. Deviations from the nominal tilt angle were extracted from the output of the program Tiltdiff (Henderson et al., 2011). The average deviation values shown in Fig. 4B indicate the mean deviation from the nominal tilt (from 165 tilt-pairs) for each model.

Fig. 4.

Evaluation of the tilt test in combination with OptiMod: (A–C) Each scoring metric used in OptiMod (y-axis) is plotted against the Fourier shell correlation (0.5 threshold) between the output model and the EMDB structure (x-axis). (A) The ability of the TILTDEV scoring metric, which describes the average deviation from the nominal tilt angle, to assess the optimal solution is displayed for the 80S ribosome and compared with (B) the CCC_PR scoring metric from Fig. 2B. (C) The combination of the two into a single criterion provides a better prediction of the true structure of the 80S ribosome.

4. Results

The performance of OptiMod is demonstrated using two experimentally derived cryo-EM data sets. First, the approach is used to determine the optimal ab initio reconstruction of the eukaryotic 80S ribosome, a quintessential and relatively straightforward test case in single-particle EM. Second, the approach is used to determine the optimal ab initio reconstruction of the transferrin–transferrin receptor (Tf–TfR) complex, a small and challenging macromolecule that has in previous studies provided multiple different, but equally plausible reconstructions depending on the manner in which the initial model was generated (Cheng et al., 2006).

Prior to initial model calculation, both experimental data sets were processed using standard single-particle procedures to obtain class averages (see Section 3). Subsequently, the class averages were input into OptiMod, and the method was run using default parameters. Specifically, OptiMod first generated 1000 raw volumes for each experimental data set. After 3D alignment and clustering, these volumes were averaged into 112 (ribosome) and 123 (Tf–TfR) 3D classes, and each 3D class was refined against the input class averages (note that in the release version of OptiMod, any stack of class averages or raw particles can be used for this refinement step). Finally, the refined models were scored using the CCC_PR criterion (see OptiMod Procedures, section 2.3). The multiple scored models output by OptiMod were compared to the known published structures of each of these complexes, which have both been previously characterized using different structural techniques (Ben-Shem et al., 2011; Cheng et al., 2006, 2004; Spahn et al., 2001). This comparison provides a metric for evaluating the performance of OptiMod and the quality of the output models. The evaluation can be done in a number of ways; the one used here was to calculate the Fourier shell correlation (FSC_0.5) using the 0.5 threshold value between each OptiMod output model and the published structures obtained from the EMDB or PDB (EMD-1076 for the 80S ribosome and PDB-1SUV for Tf–TfR). The lowest FSC_0.5 value between the calculated model and the published structure was assumed to be the best map calculated by OptiMod.

4.1. OptiMod application – 80S ribosome experimental data

The ribosome is a large macromolecular complex found in all living cells and is responsible for the production of proteins. Ribosomes have been studied by electron microscopy for a long time (Lake, 1976; Penczek et al., 1994; Radermacher et al., 1986; van Heel and Stoffler-Meilicke, 1985) and are frequently used as quintessential specimens for algorithm development (Elmlund and Elmlund, 2012; Yang et al., 2012). Despite lacking symmetry, and often including substantial heterogeneity (Ratje et al., 2010), the eukaryotic 80S ribosome represents a straightforward case for Euler angle determination (Henderson et al., 2011) and subsequent ab initio reconstruction, due to its large size (4200 kDa) and large quantity of RNA, which both contribute to high contrast features (SNR ~0.05 (Baxter et al., 2009)) in the raw images.

Class averages of the 80S ribosome from cryo-EM data exhibit a wide distribution of different views (Fig. 2A). These were used as input for OptiMod, which generated 112 refined and scored models. Each data point in Fig. 2B represents the FSC_0.5 value between one of the 112 refined maps calculated by OptiMod and the published ribosomal structure (EMD-1076)(Spahn et al., 2001). There is a high overall correlation (−0.82) between the ability of the CCC_PR metric to internally rank each model and the similarity of that model to the true solution.

Three representative volumes generated by OptiMod, corresponding to the best, middle, and worst solution (as compared to the published structures), are shown in Fig. 2D, before and after refinement against the input classes. The published map is shown in Fig. 2C for comparison. Despite a common starting point represented by well-aligned class averages, it is clear that multiple 3D solutions can be obtained even after aligning, classifying and averaging the raw models. The refinement step (procedure 4 in Fig. 1), thus serves several purposes: (1) for the best models, it restores fine details that were averaged out during initial angular assignment (example 1 in Fig. 2D); (2) for the poorer models, it can potentially aid and speed up the convergence toward the correct structure (example 2 in Fig. 2D). However, example 3 in Fig. 2D, clearly shows that not all volumes will refine to the correct structure, and determining which features are responsible for proper convergence is highly dataset and algorithm dependent. Thus, a selection criterion must be in place that serves to discriminate a good candidate model from a bad candidate model. A standard solution is to show how well the class averages (the input) fit to re-projections of the reconstructed map along the identified Euler angles (the output). We make use of a simple selection criterion (CCC_PR in procedure 5) that is based on the cross-correlation coefficient between input and output to rank the different solutions in relative terms (described in section “OptiMod Procedures”, section 2.3). Importantly, such a criterion can only discriminate the best from the worst solution internally within the data set at hand, and it cannot determine overall correctness per se without additional validation criteria (see tilt-pair section 4.3). Nevertheless, our data (Figs. 2B and 3B) indicate that it is possible to rely on this criterion to rank the models.

4.2. OptiMod application – Tf–TfR experimental data

In vertebrates, iron is transported in the serum bound to Transferrin and is delivered to cells through the Transferrin Receptor. Together, the two form a 290 kDa complex with two-fold rotational symmetry (Cheng et al., 2004). The small size (<500 kDa), preferred orientation, and relative lack of strong features makes Tf–TfR a challenging test case for single-particle analysis, and in particular for ab initio reconstruction. A previous study has explored a variety of methods for reconstructing the Tf–TfR complex and concluded that a common lines approach is prone to error and user bias and must be validated using alternative approaches (Cheng et al., 2006).

Class averages of the Tf–TfR complex from cryo-EM data exhibit a highly preferred orientation for the 5-lobed frontal view (Supplementary Fig. 2). To improve the representation of alternative views of the complex, we manually selected an approximately equal number of distinct class averages to input into OptiMod (Fig. 3A). This is not a requirement, but often facilitates initial model calculation. Optimod was used to generate 1000 raw volumes, which were automatically aligned, clustered into 123 groups, refined, and scored with the CCC_PR criterion. As in the case of the 80S ribosome, we observe a trend whereby the worst models ranked by the CCC_PR criterion represent solutions to the initial model problem that are further from the true structure of Tf–TfR (Fig. 3B). However, this trend is not as pronounced as in the ribosomal test case; the correlation between how well the CCC_PR criterion can rank the models and the similarity of that model to the true solution is lower for the Tf–TfR data set (−0.64) than it is for the 80S ribosome (−0.82) (compare Fig. 2B with Fig. 3B). This indicates that the small size of the Tf–TfR complex (290 kDa) and fewer representative features within the 2D images can present challenges not only for Euler angle assignment (Henderson et al., 2011), but also for the assessment of the resulting models. It is likely one of the reasons for the challenges that were faced in the original study when trying to obtain a satisfactory solution to the initial model problem (Cheng et al., 2006).

Three representative models from the output of OptiMod were selected to represent the best, middle, and worst solution as compared to the published map (Fig. 3D). Each is shown before and after asymmetric refinement, and additionally after automatic identification of the symmetry axis and refinement with two-fold rotational (C2) symmetry. The latter was performed because symmetry effectively decreases the angular search space and can sometimes facilitate refinement convergence. Although the five-lobed frontal view of the complex is present in nearly every initial model, the side views clearly indicate that some are closer to the correct solution than others, a behavior that has been previously observed and which has led to the conclusion that reconstructions based on common-lines require careful scrutiny and validation (Cheng et al., 2006). Indeed, C1 refinement, even after exhaustively iterating the procedure to allow for convergence (see Section 3), did not stabilize at an appropriate structure (middle row in Fig. 3D, example 2). The presence of symmetry within this particular macromolecular complex, and its appropriate specification during refinement, can enable more rapid convergence to the correct result (bottom row in Fig. 3D, example 2). However, this can also be error-prone in practice. For example, as in the case of the 80S ribosome, the worst models ranked by the CCC_PR criterion represented solutions that were unlike the true structure of Tf–TfR and could not be further refined (example 3 in Fig. 3D). The incorporation of a selection criterion thus serves to rank and discriminate the different outputs, even in challenging test cases.

4.3. Incorporation of tilt-pair data

In principle, any scoring metric can be incorporated and used in OptiMod as an external assessment, either by itself or in combination with others. To demonstrate this, we collected 165 particle tilt-pairs (Rosenthal and Henderson, 2003) of 80S ribosomes. For each clustered and refined model calculated by OptiMod, Euler angles were assigned to the tilt-pairs, as described in (Henderson et al., 2011). Subsequently, for each model we calculated the average deviation from the nominal tilt angle between all the particle tilt-pairs:

$$\mathit{TILTDEV}=\sum _i({\theta }_i)/i$$ (3)

$${\theta }_i=\sum _j(\mid {\theta }_j-{\theta }_n\mid )/j$$ (4)

where i is the number of refined models, θ_i, is the average deviation between the tilt angle assigned to the tilt-pair θ_j, and the nominal tilt angle θ_n for the set of particle tilt-pairs j. In our case i = 112; j = 165.

For each map output by OptiMod, we calculate the value for FSC_0.5 between the map and EMD-1076. Each value is represented by individual data points in Fig. 4A. As expected from previous studies (Henderson et al., 2011; Rosenthal and Henderson, 2003), we observe a high correlation between the ability of the TILTDEV criterion to indicate the correct map and the true correlation to the published structure (Fig. 4A). In the case of the 80S ribosome, the performance of the TILTDEV criterion in Fig. 4A is comparable to the performance of the CCC_PR criterion in Fig. 4B in terms of its ability to predict the correct result, but this similarity may not necessarily apply to other macromolecules.

Multiple scoring metrics can be combined into a single criterion. Although the TILTDEV and CCC_PR assessment metrics (Fig. 4) have different units and scales, it is possible to combine them and put them onto a comparable dimensionless scale by expressing each as a ratio of its deviation from its mean (ν − ν̄) with its standard deviation σ_k(ν), where the mean and standard deviation values are calculated with respect to the set of models. Thus, for each metric k, a combined R-value can be calculated:

$$R=\sum _k{\omega }_k{S}_k[({\nu }_k-\overline{\nu })/{\sigma }_k(\nu )]/\sum _k{\omega }_k$$ (5)

where the sums are taken over the full set of independent metrics, ω_k defines the weight assigned to each metric, and S_k is ±1, depending on whether the metric has to be maximized (S_k = +1, CCC_PR) or minimized (S_k = −1, TILTDEV). A similar approach has been described for the fitting of atomic coordinates into an electron density map (Rossmann et al., 2001). Here, the combination of TILTDEV and CCC_PR metrics into a single criterion improves the overall prediction (Fig. 4C). The primary value of using multiple scoring metrics is the ability to average out inconsistencies and errors arising from the use of only a single criterion, and in principle, any number of metrics can be used to assess the result. However, metric combination improves model prediction only if each independent criterion can accurately assess the true solution. In practice, this may not be true (for example, see (Chandramouli et al., 2011)). Thus, the default is to use the CCC_PR metric alone to measure internal consistency, and ideally combine with the tilt test to select for and validate the optimal model using an external measurement, provided that this data is available.

5. Discussion

Obtaining a reliable starting model in single-particle electron microscopy is a well-known problem that presents numerous challenges and to which different solutions that use common-lines based methods have been proposed (Crowther et al., 1970; Elmlund et al., 2010, 2009; Elmlund and Elmlund, 2012; Goncharov and Vainshtein, 1986; Penczek et al., 1996; Singer et al., 2010; van Heel, 1987). The OptiMod approach described here is designed to consolidate these solutions in order to automate and optimize the result of any ab initio reconstruction package that produces an initial 3D map. It aims to avoid user bias and lack of user experience by calculating ab initio reconstructions from many combinations of class averages. In doing so, it is additionally capable of sampling multiple different solutions that may potentially be obtained by the employed algorithm, each of which may or may not be relied upon as an initial model. Thus, a selection criterion that correlates the input class averages to the output re-projections is used to rank the different solutions with respect to one another in terms of overall quality. This criterion can be used by itself or in combination with tilt-pair data in order to improve the overall prediction.

The primary consideration for initial model calculation is correctness of the molecular envelope of the object, as described in (Henderson et al., 2011). When the initial model is correct, Euler angles assigned to an external set of particle tilt-pairs should always be centered about the nominal tilt angle of the stage. As long as the correct molecular envelope is established, any refinement package (Frank et al., 1996; Grigorieff, 2007; Heymann, 2001; Hohn et al., 2007; Scheres, 2012; Sorzano et al., 2004; Tang et al., 2007; van Heel et al., 1996) should be able to extract high resolution information from the data. Thus, the ultimate goal of OptiMod is to make this particular step – initial model determination – more robust, routine, and reliable.

In the current implementation, OptiMod is not designed to deal with large-scale heterogeneity. Although the term “large-scale” can be somewhat relative and depends on the manner in which the data is processed, we are specifically referring to situations where multiple clearly distinct shapes and/or sizes are present in the same data set, and in which a single starting reference is insufficient to describe and refine the full spectrum of structures. A good example is the assembly pathway of the 30S ribosomal subunit, where interpretable 3D structures can be obtained for a large range of shapes and sizes using the random-conical tilt reconstruction strategy, starting with a small sub-domain of the 30S ribosomal subunit and ending with the fully assembled specimen (Mulder et al., 2010). Minor heterogeneity is readily tolerated in OptiMod – the result is simply an averaged volume with the heterogeneous regions less well defined. In such cases, a single 3D model can be used as a starting reference to assign approximate Euler angles from which multiple classes can then be obtained by 3D classification (Lyumkis et al., 2013a; Scheres, 2012) Indeed, most 80S ribosome populations, including the one used in this study, contain multiple conformational and/or compositional states. Likewise, OptiMod would perform poorly in the presence of a large number of false positives within the data. This is because a solution will be found regardless of whether or not it is correct. The best manner in which to reduce this potential problem is to incorporate the tilt-test into the validation criteria, as demonstrated in Fig. 4 (Henderson et al., 2011; Rosenthal and Henderson, 2003). Incorporation of the tilt test will: (1) improve the selection of an optimal model, (2) provide an external measurement with which to assess its overall correctness, and (3) determine the absolute hand of the object (which cannot by definition be achieved using a common lines routine). Additionally, the tilt test may help to understand if a model corresponds to a distinct solution from a heterogeneous mixture of populations, or simply an inappropriate solution to the initial model problem in general (e.g. Figs. 2 and 3). Finally, it is worth noting that any common-lines based method will fail if the particle of interest exhibits a single, preferred orientation. In such cases, the random-conical tilt reconstruction approach would be better suited to obtain one, or several, 3D maps of the macromolecule of interest (e.g. (Brignole et al., 2009; Lyumkis et al., 2013b; Radermacher et al., 1986)).

We describe steps that can be taken in order to maximize the success of obtaining a reliable initial model for high-resolution single-particle based refinement. The most important consideration is the acquisition of a set of class averages that is representative of the data, which can be achieved using an ab initio approach for alignment and clustering. The goal of any such procedure is to identify the predominant projection views of the imaged objects by extracting and bringing into register subsets of similar images within the data. In principle, any ab initio alignment and clustering approach can be used (Ogura et al., 2003; Scheres et al., 2005; Sorzano et al., 2010). However, the selection of the particular algorithm can potentially impact the resulting initial model. Particle misclassification, whereby the noisy images are improperly grouped into a cluster sum that is poorly representative of the underlying data (e.g. (Shaikh et al., 2008)), can be a common problem and is algorithm-specific. Recently, we have obtained particularly good results with the Iterative Stable Alignment and Clustering (ISAC) algorithm (Yang et al., 2012), in part because it contains a strategy to evaluate stability and reproducibility of the alignment solution and remove particles that are not consistently representative of the data. Both datasets described here have been processed using the ISAC algorithm. Once the class averages are obtained, it is useful to parse through them and select a subset containing an approximately equal distribution of different views. This step is particularly relevant for particles with preferred orientation (e.g. compare Fig. 3A and Supplementary Fig. 2). In practice, one rarely requires more than ~50 class averages for the raw volume calculation (Fig. 1, procedure 1), but the use of many more class averages for the refinement (Fig. 1, procedure 5) is advantageous. OptiMod allows the user to define multiple sets of class averages for these two distinct steps. As described in the “Algorithm Description” section, it is often useful to align the class averages to each other or center the class averages prior to angular assignment. Both steps can be performed internally inside OptiMod. Typically, several simultaneous jobs are then run on the same set of input class averages, with and without the pre-processing step, and the results of the refined volume assessment are compared across jobs. Should the jobs provide unsatisfactory results, the most useful solution in our hands has been to go back to the class average generation, selection, or pre-processing stage, and secondarily to modify parameters listed in Table 1. The above strategies are in routine use for ab initio 3D reconstruction at the National Resource for Automated Molecular Microscopy (nramm.scripps.edu).

OptiMod is fully integrated into the Appion pipeline for image processing and makes use of EMAN1, Xmipp, and IMAGIC (optional) processing packages. A standalone version is made available through a set of python scripts, assuming the existence of underlying dependencies (python 2.4 or newer, EMAN1, and Xmipp). All scripts are freely available under the Apache Open Source License, Version 2.0. Software can be downloaded from http://ami.scripps.edu.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jsb.2013.10.009.