A practical look at cryo-electron tomography image processing: Key considerations for new biological discoveries

Abstract

Cryo-electron tomography (cryo-ET) enables 3D visualization of complex biological environments without the need for purification, thereby preserving the native biological context of the specimen. For determining macromolecular structures, repeating molecules can be localized in tomograms and subjected to subtomogram averaging, the 3D analog to single particle analysis. In addition to molecular structure, tomograms have a wealth of other information that can be accessed through image processing, including the analysis of membrane surfaces, cytoskeletal filaments, and the relationships between molecules of interest. Here, we provide an overview of recent developments in cryo-ET image processing with the goal of clarifying key considerations to help new users obtain novel biological findings.

Introduction

Cryo-electron tomography (cryo-ET) is a cryo-electron microscopy (cryo-EM) method where the stage is tilted around a target position while a series of images is collected. Tilting provides different perspectives of the field of view, enabling the reconstruction of a 3D representation called a tomogram. By increasing dimensionality, overlapping biological structures that are obfuscated in projections can be separated and analyzed. This makes cryo-ET uniquely capable of analyzing complex biological systems under near-native conditions that preserve biological context.

Subtomogram averaging (STA) allows for high-resolution structure determination of repeating molecules in tomograms [1,2]. While it is attractive to imagine that cryo-ET will enable visualizing every molecule in near-native environments at atomic resolutions, there are major barriers that prevent this from being a practical reality. Due to electron damage, the number of particles required for STA is, at minimum, the same as for single particle analysis (SPA). Cryo-ET samples are typically thicker than SPA specimens, which reduces the signal-to-noise ratio (SNR), thereby increasing the number of requisite particles. Compared with SPA images, molecules in cellular sections often have a small number of copies per tomogram. This can also be exacerbated by higher conformational or constitutional heterogeneity within the cell as compared with biochemically purified molecules, though complexes that are difficult to purify or reassemble in vitro, such as nuclear pore complexes [3], may still be better studied in cells. Cryo-ET data acquisition is also significantly slower than SPA, which can be further hindered by the time and expense of generating lamella by focused ion beam (FIB) milling [4] as well as the potential need for correlative light and electron microscopy [5].

Despite this myriad of challenges, it has been clear that even in the post-resolution revolution era, modestly resolved cryo-ET structures can be combined with other methods to provide novel biological insights on otherwise intractable systems (Figure 1a) [6]. After STA, orientational metadata can also be used to study the native biological context (Figure 1b) [7]. Segmentations (Figure 1c) [8] and morphological analyses (Figure 1d) [9] of tomograms can also provide novel biological insights. In this manuscript, rather than outlining the current slate of image processing algorithms, which has been well reviewed in recent publications [1,2], I will provide an outline of the cryo-ET imaging processing workflow with practical considerations on how to solve biological problems.

Figure 1. Recent examples of different cryo-ET approaches applied to biological problems.

(a) STA structure of the SARS-CoV-2 nsp3–nsp4 pore complex determined to 4.2 Å resolution [6]. This map is at a sufficient resolution for molecular modeling and one of a few STA structures at this resolution range that is not of a high-abundance discreet protein or pleomorphic assembly. It is important to consider that this 3 MDa complex has a C6 symmetry and still required 4635 tomograms. (b) A model of human heterochromatin assembly at the nuclear periphery [7]. Template matching and STA were used to identify and determine the orientation of nucleosomes; this metadata was used in molecular simulations to build a model of the chromatin network in its native context. This is a strong example of the usefulness of particle metadata and hybrid approaches. (c) Automated segmentation of a SARS-CoV2-infected cell using Ais [8]. Double membrane vesicles (DMVs), light red; single membranes, gray; viral nucleocapsid proteins, red; viral pores in DMVs, blue; nucleic acids in DMVs, pink; microtubules, green; actin, cyan; intermediate filaments, orange; ribosomes, magenta; and mitochondrial granules, yellow. This is an example of the types of automated segmentations and annotations that are possible with current AI algorithms. (d) Visualization of mitochondrial membrane distance and associated ribosomes [9]. In the left and center panels, regions where the distance between the outer mitochondrial membrane (OMM) and inner mitochondrial membrane (IMM) are <10 nm are shown in blue and regions >10 nm are shown in gray. Ribosomes oriented for import to the OMM are in blue while remaining ribosomes are in pink. Right inset colored by OMM-IMM distance. This study shows how quantitative comparisons of particle orientations and membrane can provide information for structural classification as well as new insights into membrane biology. cryo-ET, cryo-electron tomography; STA, subtomogram averaging.

On specimen preparation and data acquisition

Each step of the cryo-ETworkflow builds on top of previous ones, so suboptimal specimens or data collection will fundamentally limit any downstream processing. Here, I briefly mention some useful considerations.

Specimens

For purified assemblies like viruses or vesicles, specimen preparation typically consists of purification and plunge freezing, similar to SPA specimens. While the thinnest ice possible is desired in SPA, over-blotting of pleomorphic assemblies can crush or distort them, so some trade-offs are required.

While FIB-milling is the de facto approach for cellular specimen preparation, it does have limitations that affect image processing such as damage at the lamella surfaces [10–12], and specimen charging [13]. As such, it can be preferable to avoid FIB-milling by either using smaller cells or looking at the periphery of flatter cells. Lamella can only be FIB-milled from central slices, which can limit the targeting of certain biological features. In these cases, alternative approaches such as waffle milling [14], lift out [15,16], or unroofing [17] may be required.

Prior to the adaption of FIB-milling to biological materials, cryo-electron microscopy of vitreous sections (CEMOVIS) was the main approach to thin cells for cryo-ET [18]. CEMOVIS uses a cryo-microtome to section cells, which can be higher throughput than FIB-milling, and allows for serial sectioning. However, CEMOVIS typically requires significantly more dexterity to successfully perform [18] and is prone to a variety of cutting artifacts due to the use of a physical knife [19]. While such artifacts may limit the accuracy of contextual analysis of molecules within the cell, it has recently been shown that molecular structures are well preserved [20], which may make it an attractive approach when particle count, and not cellular context, is the main concern.

Data acquisition

Parameters for tilt series acquisition include the tilt range (how far to tilt), the tilt step (the spacing between tilts), and the tilt scheme (how to tilt the stage during collection) [21]. The first two parameters determine the distribution of sampled and missing information in the 3D reconstruction, i.e. the “missing wedge,” and subsequent distortions in the tomogram [22,23]. The tilt scheme determines how the dose, and resultant beam-induced distortions, are distributed across the tilt series [21]. The dose-symmetric tilt scheme provides an optimal dose distribution at the cost of increased acquisition time, though recent multi-shot approaches can ameliorate this by enabling the collection of multiple tilt series in parallel [24–26]. Additionally, tilt series can be collected as montages [27], where multi-shot tilt series are collected with overlapping regions and stitched together, allowing for the reconstruction of continuous tomograms over large fields of view. This results in increased exposure at the overlapping regions, though recent approaches have been developed to minimize this effect [28,29]. Nevertheless, this over-exposure in regions of the tomogram may result in poorer resolution during STA, so the user must determine if this is a reasonable tradeoff for the large fields of view.

The tilt series geometry is balanced against the total dose budget, the specimen-dependent number of electrons that can be tolerated before noticeable damage occurs. Here, damage is not just important in regards to the loss of high-resolution signal [30], but also in causing bubbling or distortions to the specimen [31] that limit the ability to align tomograms and the quality of tomographic reconstructions. Sensitivity to radiation damage depends on the types of biological molecules [31] and the composition of these molecules within the field of view, which varies between cell types and subcellular regions. As such, the tolerable dose generally cannot be determined a priori but can be roughly estimated by imaging your specimen until degradation occurs; a typical range is 100–140 e/Å^{^}2. Dividing the total dose by the number of tilt images provides the dose per image. More images results in lower dose per image, reducing SNR and tilt-series alignment accuracy, though this is a necessary tradeoff if your biological problem requires this. The Crowther criterion demonstrates that finer angular sampling provides more complete information with respect to resolution [32]; the practical effect of this is that smaller particles have improved contrast with smaller tilt increments. Smaller particles such as nucleosomes [7] or assemblies with smaller subunits, such as tubulin [33], often benefit from 2° tilt increments rather than the 3° increments typically used for ribosomes [34] or viral assemblies [35]. Recent methods to interpolate intermediate tilt images may circumvent the need for smaller tilt steps in some processing tasks [23].

Other parameters to consider are magnification and dose per frame. There are a number of factors that can limit the achievable resolution by STA, including a larger number of interpolation steps compared with SPA, errors in tilt series alignment due to local deformations (see Structure Determination Methods), or specimen-related issues such as FIB-related damage [10–12]. Empirically, few deposited structures in the Electron Microscopy Data Bank (EMDB) reach Nyquist resolution, but a useful rule of thumb can be to estimate the resolution limit of the dataset as 3x the pixel size. The dose per frame affects the performance of frame alignment; another rule of thumb is to aim for 0.5 e/pix/frame (the key units here are pixels, notÅ). Frame time is calculated from the dose rate and dose per frame.

Data preprocessing and tomogram reconstruction

Preprocessing spans from data acquisition to tomographic reconstruction [1,2]. This includes frame alignment, exposure filtering, contrast transfer function (CTF) estimation and correction, and tilt series alignment. Given the serial nature of cryo-ET processing, the quality of preprocessing greatly influences downstream analysis. While some aspects of preprocessing such as CTF estimation and tilt series alignment can be refined after STA (see Structure Determination Methods), such reference-based methods often rely on medium-resolution structural features in averages, ideally in the sub-nanometer range, which is still challenging for many specimens. As such, optimal preprocessing remains a key factor in obtaining sub-nanometer resolution averages; refinement is no substitute for suboptimal preprocessing.

Motion correction

The algorithms and software used for motion correction in cryo-ET are the same as in SPA. In practice, the choice of software may be more dependent on your workflow; some users prefer on-the-fly correction using software like AlignFrames in SerialEM [27], while others prefer offline processing.

Exposure filtering

Exposure filtering is an approach for low-pass filtering images based on the cumulative electron exposure [30]. This prevents the projection of high-resolution noise from high-tilt images during reconstruction, improving the contrast in tomograms. We typically perform this step prior to tilt series alignment as filtering can be useful for fiducial-less alignment approaches.

Tilt series alignment

Alignment is the process of determining the geometric parameters required to reconstruct tomograms. Arguably, the standard package for tilt series alignment remains IMOD [36], which allows for fiducial-based alignment or image-based patch tracking. This is due to its semiautomated usage, where users can supervise and adjust parameters for each alignment step and tilt series; this flexibility comes at the cost of substantial user effort. With increasingly large datasets, fully automated tilt series alignment is often a more practical approach. While, IMOD can also be run in a fully automated batch processing mode, newer fiducial-based approaches in Dynamo [37] and fiducial-less approaches in AreTomo [38] are becoming widely adopted.

CTF estimation and correction

The CTF is a defocus-dependent aberration that distorts information and fundamentally limits resolution, so accurate CTF estimation and correction are essential. Tilting causes a continuously varying CTF in images [39], making CTF estimation in cryo-ET particularly difficult; this is further exacerbated by low SNR. Recently, several CTF estimation approaches have been developed that make direct use of tilt-series alignment information [40,24,41,42,39], which significantly improves estimation accuracy.

CTF correction typically occurs concurrently with tomographic reconstruction. Approaches can be 2D [43], which accounts for the defocus gradient across tilted images, or 3D [44], which also accounts for the thickness of the specimen. While 3D CTF correction is more precise and significantly improves the resolution of STA [44], it comes with increased computational cost. Reference-based refinement approaches (see Structure determination methods) typically reconstruct averages from tilt series data, so unbinned CTF-corrected tomograms may not be necessary for high-resolution averaging. However, binned CTF-corrected tomograms can still be useful for earlier steps like template matching or initial STA.

Tomogram reconstruction

Tomograms can be reconstructed with various algorithms, which affect their appearance and utility [22,45]. Algorithms that directly use the image data for reconstruction are the most accurate and are best suited for STA; these include weighted back project (WBP) or Fourier inversion. Historically, WBP is the mostly widely used, in part due to the reduced memory requirements compared with Fourier transforming a whole tomogram. While the Fourier slice theorem implies Fourier inversion is an equivalent operation to WBP, for slab-like specimens, larger areas of the specimen to enter the field of view during tilting, making the total information content in each Fourier slice different; this results in more pronounced edge artifacts [22]. Recent tile-based approaches address aspects of the memory and edge artifact issues [46,47] but they do not directly address 3D CTF correction (see CTF estimation and correction), which is simpler with real space WBP [44] than with Fourier inversion [48].

Algebraic approaches, such as Simultaneous Iterative Reconstruction Technique (SIRT) or Simultaneous Algebraic Reconstruction Technique (SART), reconstruct tomograms indirectly by solving sets of linear equations to calculate voxel gray values in the tomograms that best match the gray values from all contributing projections [45]. By solving for gray values that effectively represent the consensus of the input projections, tomograms reconstructed by algebraic methods typically have less noise and higher contrast but lack high-resolution information, which is not sufficiently strong in all projections. As such, it is common to reconstruct multiple tomograms for each tilt series; direct methods are used for STA while algebraic methods are used for situations where contrast is important but resolution is not, such as visual analysis or segmentation.

In addition to algebraic reconstruction approaches, filtering is also a common method of improving contrast in tomograms [22,49]. Recently, machine learning-based approaches have been developed for enhancing tomograms by denoising or missing wedge compensation [50–52]; while not explicitly for contrast enhancement, they have a similar effect of increasing the visual interpretability of tomograms. The performance of algebraic reconstructions, filtering, or machine learning approaches for improving the visual quality of tomograms is heavily dependent on the specimen [53], so testing is required to determine the optimal approach for your data.

Particle picking

Prior to STA, repeating particles must be identified and localized; we will refer to this as particle picking [1], though these do not need to be discreet particles and can include asymmetric units of larger assemblies.

Discrete particles

We define discrete particles as molecules that are free in solution. The most widely used method for picking discrete particles is template matching, where an experimentally determined or simulated EM density map is used to detect particles in a tomogram by cross-correlation; this can provide the position and rough orientation of molecules. Template matching has had a recent resurgence, with algorithmic improvements providing higher accuracy and GPU implementations greatly reducing the computational cost [54–56].

Newer approaches include machine learning-based algorithms [8,57–61] and a novel tensorial template matching algorithm [62]. While machine learning-based algorithms do not yet seem to have the broad accuracy of template matching across specimens, they have the promise of significantly lower computational costs and for some algorithms, the ability to pick particles with no a priori information. Tensorial template matching has not yet been well validated in the field but has shown promising results. Note that these approaches detect particles; they do not provide orientational information.

Surface-associated particles

Surface-associated particles include molecules within larger assemblies, such as membrane-tethered proteins, viral subunits, or cytoskeletal filaments. While methods for discrete particles can be applied to surface-associated particles, this can result in the loss of useful information. Surface-associated particles are oriented relative to the surface, providing a strong prior that reduces the orientational search space. The surfaces themselves are often of interest, so metadata linking the particle to the surface is also important.

For regular surfaces such as spheres or filaments, particles can be manually defined by simple geometric functions; e.g. a point and radius for spheres or splines and radii for filaments. For irregular surfaces, segmentation (see below) may be necessary. In either case, the particle positions on the surfaces are typically not known. As such, it is often useful to oversample the surface and use STA to identify true positions. An exception to these generalizations is PySeg [63], which uses discrete Morse theory to trace networks from tomogram gray values; graphs from these networks are then used to detect membrane bound complexes without the need for segmentations while also providing initial normal vectors.

Many groups code their own scripts or tools for regular surfaces, but a number of openly distributed tools are now available. Dynamo [64] has long had a complete set of GUI-based tools but recent plugins to visualization packages such as Napari [65,66] or the AritaX plugin [67] for ChimeraX [68] have further made such approaches more accessible. Other segmentation approaches are described below.

Structure determination methods

STA determines structures by using “subtomograms”, particle-containing volumes cropped from tomograms. The main goal is to iteratively determine the orientation of each particle by aligning subtomograms to a reference structure and averaging the top scoring orientations to yield a new reference for the next iteration of alignment. Some of the aforementioned picking approaches provide rough orientations, so STA often requires only a local orientational search. Refer to Refs. [1,2] for detailed reviews on STA algorithms and packages.

Single particle tomography (SPT) is where 2D projections centered on particles are cropped from tilt series, i.e. a subtilt series [41,66]. Alignment of subtilts is similar to SPA but can include constraints to link the projections of each particle. Since tomographic reconstruction is not required and subtilt series are smaller than subtomograms, SPT has significantly reduced computational costs. The limitation of SPT is that overlapping information is not separated in the subtilts, which can impact the ability to align particles, particularly at low resolutions. As such, it can be useful to perform initial alignment by STA before SPT.

With sufficiently large datasets, STA resolution is often limited by errors in CTF estimation or tilt series alignment related to local deformations in the tilt series. While global parameters are determined during preprocessing, averaged structures can be used to refine these parameters locally [41,66,69,70]. Averages can be used to localize the particles in the tilt series; these positions can be used as fiducial markers to refine the tilt series alignment. Averages can also be projected and compared with the raw particle projections to refine orientations and CTF parameters, with or without considering local deformations. As mentioned above (see Data preprocessing and tomogram reconstruction), such refinement approaches are often driven by sub-nanometer resolution details in the averages, which can be difficult to achieve without optimal preprocessing; refinement thus builds upon parameters determined during preprocessing and cannot rescue suboptimal preprocessing.

Segmentation, morphometrics, and metadata analysis

While structure determination is an important part of cryo-ET data analysis, it is far from the only use. The unique information provided by cryo-ET is biological context, which includes information about molecules like their localization within different regions of the cell, affinity with membranes, or association with other molecules.

Segmentation

Segmentation is the process of defining continuous structures like membranes or cytoskeletal filaments [49]. Segmentation is particularly important in cellular cryo-ETas it can localize virtually everything that isn’t a discreet molecule and often defines cellular compartmentalization. Recently, machine learning approaches have revolutionized segmentation [8,71,72], providing automated approaches on par with or better than manual approaches. Additional tools use segmentations to visualize membrane surfaces [73,74], which can aid in picking and analyzing membrane-associated complexes. As with other machine learning approaches, the broad applicability of packages is often limited by the training data used, so it is best to test different packages and determine what works best for your own data.

Morphometrics

Segmentation typically provides voxel-specific surface information encoded in 3D maps. Converting this voxel data into surface or graph representations can provide a wealth of information for quantitative analysis [75]; this is referred to as morphometrics. Morphometric analyses in cryo-ET are still in relatively early days but recent pipelines have illustrated its particular usefulness in deriving new biological insights.

Metadata analysis

While morphometrics refers to the analysis of segmentations or segmentations and particles, there is also important biological information encoded in the relative orientations of particles with each other. Examples include measuring spatial distributions of particle density [76], analyzing the relationships between neighboring molecules [77], tracing irregular networks of molecules [7], and characterizing heterogeneity in geometric assemblies [78]. Historically, this type of metadata analysis has been performed using lab-specific ad hoc scripts, but user-friendly packages are currently under development [7].

Discussion

Cryo-ET has become an important structural biology method due to its ability to provide molecular-scale information from near-native specimens, particularly cellular sections. While structure determination is important, it is arguably the least powerful aspect of cryo-ETas high-resolution structures are generally much easier to obtain by other methods and will continue be so for the foreseeable future. Instead, the truly novel insights that only cryo-ET can provide also leverage native biological contexts such as compartmentalization or molecular relationships within the cell. This is a particularly exciting time as there is a rapid expansion of approaches to meet the challenges of segmentation, morphometrics, and metadata analysis, particularly with the adoption of machine learning methods.

Despite this wealth of new methods, significant challenges remain. Key among them is how to combine these disparate packages into coherent workflows. Packages including TomoBEAR, TOMOMAN, and ScipionTomo help facilitate this [53,79,80], but given the rate of new software packages, significant user scripting will be required for a while. Given this rapidly changing and expanding field, I hope that this manuscript will serve as a general guide for how cryo-ET image processing approaches can yield new biological insights.

Data availability

No data was used for the research described in the article.