GENFIRE: A generalized Fourier iterative reconstruction algorithm for high-resolution 3D imaging

Abstract

Tomography has made a radical impact on diverse fields ranging from the study of 3D atomic arrangements in matter to the study of human health in medicine. Despite its very diverse applications, the core of tomography remains the same, that is, a mathematical method must be implemented to reconstruct the 3D structure of an object from a number of 2D projections. Here, we present the mathematical implementation of a tomographic algorithm, termed GENeralized Fourier Iterative REconstruction (GENFIRE), for high-resolution 3D reconstruction from a limited number of 2D projections. GENFIRE first assembles a 3D Fourier grid with oversampling and then iterates between real and reciprocal space to search for a global solution that is concurrently consistent with the measured data and general physical constraints. The algorithm requires minimal human intervention and also incorporates angular refinement to reduce the tilt angle error. We demonstrate that GENFIRE can produce superior results relative to several other popular tomographic reconstruction techniques through numerical simulations and by experimentally reconstructing the 3D structure of a porous material and a frozen-hydrated marine cyanobacterium. Equipped with a graphical user interface, GENFIRE is freely available from our website and is expected to find broad applications across different disciplines.

Introduction

Tomography has found widespread applications in the physical, biological and medical sciences^1–7. Electron tomography, for example, is experiencing a revolution in high-resolution 3D imaging of physical and biological samples. In the physical sciences, atomic electron tomography (AET) has been developed to determine the 3D atomic structure of crystal defects such as grain boundaries, anti-phase boundaries, stacking faults, dislocations, chemical order/disorder and point defects, and to precisely localize the 3D coordinates of individual atoms in materials without assuming crystallinity^{1, 8–12}. The atomic coordinates measured by AET have been used as direct input to density functional theory calculations to correlate crystal defects and chemical order/disorder with material properties at the single atomic level¹³. In the biological sciences, single-particle cryo-electron microscopy (EM) has been applied to achieve near atomic resolution of purified protein complexes^{2, 7, 14–16}, and cryo-electron tomography allows for 3D imaging of pleomorphic samples such as viral infection mechanisms of cells with resolutions on the order of a few nanometers^17–19. These advances are not limited to electron tomography. Tomographic implementation of synchrotron X-ray absorption and phase contrast imaging has also found interdisciplinary applications^{5, 20–25}. Using the brilliance of advanced X-ray sources, coherent diffractive imaging (CDI) methods²⁶ have been combined with tomographic reconstruction for 3D quantitative imaging of thick samples with resolutions in the tens of nanometers^27–33.

Presently, a popular tomographic reconstruction method is filtered back projection (FBP)^2–4. FBP works well when there are a large number of projections with no missing data. However, when the data is inadequately sampled due to the radiation dose and geometric constraints, it suffers from artifacts, potentially clouding interpretability of the final reconstruction. This difficulty can be partially alleviated by real-space iterative algorithms such as the algebraic reconstruction technique (ART)³⁴, simultaneous ART (SART)³⁵ and simultaneous iterative reconstruction technique (SIRT)³⁶. However, these algorithms do not fully exploit the correlated information among all the projections as the iteration process is implemented through local interpolation in real space. In contrast, Fourier-based iterative algorithms use information in both real and Fourier space as part of the iterative process^{13, 37, 38}. A major advantage of these algorithms is that changes made in one space affect the other space globally. Equal slope tomography (EST)³⁷, an example of such an algorithm, has been successfully applied in AET to reconstruct the 3D arrangement of crystal defects in materials, including recovery of Bragg peaks in the missing wedge direction^{1, 8–10}. Additionally, EST was shown to produce reconstructions comparable to modern medical CT techniques but using significantly lower radiation dose^{20, 22, 39}. However, the drawback of EST is the requirement that the tilt angles must follow equal slope increments along a single tilt axis, which limits its broader applications.

Very recently, a generalized Fourier iterative reconstruction algorithm (GENFIRE) has been reported for high-resolution 3D imaging from a limited number of 2D projections¹³. GENFIRE first pads zeros to each 2D projection and calculates its oversampled Fourier slice^{40, 41}. The oversampled Fourier slices are used to accurately compute a small fraction of points on a 3D Cartesian grid based on gridding interpolation^{42, 43}. The remaining grid points that cannot be determined with sufficient accuracy are defined as unknown. The algorithm then iterates between real and reciprocal space and enforces constraints in each space. In real space, the negative valued voxels and the voxels in the zero-padding region are set to zero. In reciprocal space, the small fraction of the known grid points are enforced in each iteration, while the unknown grid points are recovered by the iterative process. After several hundred iterations, the algorithm converges to a structure that is concurrently consistent with the measured data and the physical constraints. Furthermore, GENFIRE implements an angular refinement routine to reduce the tilt angle error and can be adapted to any tomographic data acquisition geometry. In this article, we present the mathematical implementation of the GENFIRE algorithm. Using both physical and biological samples, we demonstrate that GENFIRE produces superior 3D reconstructions relative to several other tomographic reconstruction algorithms.

Methods

Assembling a 3D Fourier grid with oversampling

GENFIRE first assembles a rectangular 3D Fourier grid from a set of measured 2D projections. According to the Fourier slice theorem, the Fourier transform of each 2D projection represents a plane slicing through the origin of the Fourier transform of a 3D object^{3, 4}. Care should be taken that the Fourier slice theorem requires each 2D image to be a linear projection of the 3D object at a given angle. This requirement may not be satisfied in some experimental cases due to the presence of nonlinear effects, which will be discussed in later sections. To obtain a Fourier grid point, $F_{obs}(\vec{k})$, we compute its perpendicular distance to the Fourier plane, where D _j represents the perpendicular distance and $\left(u_j,v_j\right)$ the foot of the perpendicular line to the j ^th projection. Since ($u_j,v_j$) are not integer coordinates, we use the discrete Fourier transform (DFT) to compute the value of ($u_j,v_j$). The use of the DFT to explicitly compute the ($u_j,v_j$) value is more accurate than interpolating from the set of 2D FFTs of each projection at the cost of being computationally slower. A faster, but less accurate, FFT gridding method is also provided as an option in the GENFIRE package. After calculating the ($u_j,v_j$) values for all the projections, we determine the value of the Fourier grid point by

$$F_{obs}\left(\vec{k}\right)=\sum _{\left\{j|D_j<D_{th}\right\}}\frac{D_j^{-1}}{{\sum }_{\left\{j|D_j<D_{th}\right\}}{D}_j^{-1}}\sum _{x=-\frac{N}{2}}^{\frac{N}{2}-1}\sum _{y=-\frac{N}{2}}^{\frac{N}{2}-1}{f}_{obs}^j(x,y)e^{\frac{-2\pi i\left(x{u}_j+y{v}_j\right)}{NO}}$$ 1

where D _th is a predefined threshold, $f_{obs}^j(x,y)$ is the j ^th 2D projection with a size of N x N pixels and O is the linear oversampling ratio^{40, 41}. By properly choosing D _th and O, we accurately determine a small fraction of the Fourier grid points, while the remaining grid points are defined as unknown. In the GENFIRE package, the default value for D _th and O is 0.5 and 3, respectively.

The Fourier based iterative algorithm

Due to radiation dose and/or geometric constraints, it is desirable in many tomography applications to achieve high-resolution 3D imaging from a limited number of projections. As a result, a significant amount of the assembled Fourier grid points remain unknown after the gridding process. To recover the unknown grid points, GENFIRE iterates between real and reciprocal space with general constraints enforced in each space (Fig. 1). For the 1^st iteration, the values of the unknown grid points can be assigned to zero, random numbers or some other pre-determined numbers as the algorithm is not very sensitive to the initial input. The i ^th GENFIRE iteration consists of following five steps (Fig. 1).

• (i)Apply the inverse FFT to $F_i(\overrightarrow{k})$ and obtain the i ^th image, ${\rho }_i(\overrightarrow{r})$.
• (ii)
  Modify the image by applying the following constraints

  $${\rho }_i^{\prime }(\overrightarrow{r})=\{\begin{array}{ll}0& (\overrightarrow{r}\notin S)\cup ({\rho }_i(\overrightarrow{r})<0)\\ {\rho }_i(\overrightarrow{r})& Otherwise\end{array}$$ 2

  where S represents a support, separating the zero-padding region from the sample structure. The zero-padding region is due to oversampling^{40, 41}. This step sets the voxels outside the support or negative valued voxels inside the support to zero, while retaining the values of the other voxels.
• (iii)Apply the FFT to ${\rho }_i^{\prime }(\overrightarrow{r})$ to obtain $F_i^{\prime }(\overrightarrow{k})$.
• (iv)
  Compute $F_{i+1}(\overrightarrow{k})$by enforcing the Fourier space constraint,

  $$F_{i+1}(\overrightarrow{k})=\{\begin{array}{ll}{F}_{obs}(\overrightarrow{k})& Knownvoxels\\ F_i^{\prime }(\overrightarrow{k})& Unknownvoxels\end{array}$$ 3

  $F_{i+1}(\overrightarrow{k})$ is used for the (i + 1)^th iteration.

• (v)Calculate two R-factors, $R_k$and $R_{free}$,

$$R_k=\frac{{\sum }_{{\overrightarrow{k}}_{known}}|F_{obs}(\overrightarrow{k})-F_i(\overrightarrow{k})|}{{\sum }_{{\overrightarrow{k}}_{known}}|F_{obs}(\overrightarrow{k})|}$$ 4

$$R_{free}=\frac{{\sum }_{{\overrightarrow{k}}_{withheld}}|F_{obs}(\overrightarrow{k})-F_i(\overrightarrow{k})|}{{\sum }_{{\overrightarrow{k}}_{withheld}}|F_{obs}(\overrightarrow{k})|}$$ 5

where ${\vec{k}}_{known}$ represents the known voxels and ${\vec{k}}_{withheld}$ is a small number of randomly selected known voxels that are not used in the reconstruction. $R_k$is an error metric to monitor the convergence of the iterative process, while $R_{free}$ is an unbiased free parameter to evaluate the reconstruction, which is used in crystallography⁴⁴. $R_{free}$ is always larger than $R_k$, but for a good reconstruction the two R-factors should be consistent. Significant deviation could indicate overfitting. The algorithm is reliable and usually converges within several hundred iterations.

Figure 1.

The GENFIRE algorithm. GENFIRE first computes oversampled Fourier slices from a tilt series of 2D projections. The oversampled Fourier slices are used to accurately calculate a small fraction of points on a 3D Cartesian grid based on gridding interpolation. The algorithm then iterates between real and reciprocal space. The support and positivity constraints are enforced in real space, while the small fraction of grid points corresponding to the measured data are enforced in reciprocal space. Error metrics are used to monitor the convergence of the iterative process. After several hundred iterations, the algorithm converges to a 3D structure that is concurrently consistent with the measured data in reciprocal space and the physical constraints in real space.

In the GENFIRE package, there is also an option to use resolution extension/suppression. For experimental data, the signal-to-noise ratio decreases with the increase of the spatial frequency. To compensate the high noise level at the high spatial frequency, we implement a resolution extension/suppression technique capable of partially decoupling signal and noise through a simple modification of the way the Fourier constraint is applied. For the first iteration, only the lowest spatial frequency information is enforced. As iterations progress, higher spatial frequency data is gradually applied. This continues, forming the extension step, until half of the total number of iterations has been completed, at which point all measured data is enforced. The process is then reversed for the second half of the reconstruction, and the spatial resolution of the enforced data is gradually reduced to form the suppression step until the final iteration when only the lowest frequency information is constrained once again. While resolution extension has been implemented before^{45, 46}, to our knowledge, resolution extension/suppression has not been previously reported. We have performed extensive numerical simulations and observed that this technique can consistently improve the 3D reconstruction with noisy data (Supplementary Fig. 1). Full exploration of the generality of resolution extension/suppression requires follow-up studies.

Angular refinement

The experimentally measured tilt angles may not always coincide with the true orientations of the projections. This could be the result of many causes including instrument misalignment, slipping, beam-induced motion, vibration, thermal effects, or software error. To achieve high-resolution 3D reconstruction, we implement an angular refinement procedure to reduce the tilt angle error, which consists the following four steps.

• (i)An initial 3D reconstruction is computed using the experimentally measured tilt angles.
• (ii)For the j ^th projection, a series of 2D projections are calculated from the 3D reconstruction by varying the three Euler angles: $\phi \in [{\phi }_j-\delta \phi ,{\phi }_j+\delta \phi ]$, $\theta \in [{\theta }_j-\delta \theta ,{\theta }_j+\delta \theta ]$, $\psi \in [{\psi }_j-\delta \psi ,{\psi }_j+\delta \psi ]$, where $({\phi }_j,{\theta }_j,{\psi }_j)$ are the current best fit for the Euler angles of the j ^th projection, and (δφ, δθ, δψ) are the user-defined values for the angular search ranges. Each calculated 2D projection is then compared with the corresponding measured projection, $f_{obs}^j(x,y)$, and a quality-of-fit metric is computed. The quality-of-fit metric can be implemented by either the normalized cross-correlation or the real space R-factor. For the latter, additional translational alignment between two projections have to be performed, whereas using cross correlation the translational search is performed simultaneously. The three Euler angles with either the largest cross correlation or smallest R-factor are recorded as the refined angles for the j ^th projection.
• (iii)Repeat step (ii) for all the projections and a series of the refined angles are obtained.
• (iv)Obtain a new 3D reconstruction with the refined angles for all the projections.
• (v)Repeat steps (ii) – (iv) until no further improvement can be made.

In practice, each projection is refined in parallel, and the calculation of 2D projections from the 3D reconstruction represents the bulk of the computation. This calculation is expedited by applying the FFT to obtain an oversampled Fourier transform from the 3D reconstruction. Central slices are computed from the 3D Fourier transform using the C++ library splinterp for multithreaded linear interpolation. The inverse FFT is used to invert the central slices to the corresponding 2D projections. Care should be taken that while GENFIRE’s reconstruction can find a global minimum, the current angular refinement approach may be trapped into local minima. Further developments are needed to search for a global minimum for angular refinement.

Results

Numerical simulations on the reconstruction of a biological vesicle

Numerical simulations on the 3D reconstruction of a 64 × 64 × 64 voxel vesicle model (Fig. 2a–c) were performed using GENFIRE, EST, FBP and SIRT. Simulated projections were obtained by first calculating 2D Fourier slices of the 3D model for given angles. The corresponding real-space projections were then computed by applying the inverse FFT to the Fourier slices. This code is also included in the GENFIRE package and can be accessed graphically using the Projection Calculator. To evaluate the performance of various reconstruction algorithms with noise, we calculated 71 projections with the tilt angles ranging −70.1° to +70.1°. Noise was added to the projections at levels similar to that observed in cryo-EM images of cellular structures. Each set of projections were reconstructed using GENFIRE, EST, FBP and SIRT. The EST and GENFIRE reconstructions were performed using a loose support, the positivity constraint and 250 iterations. The SIRT reconstruction was achieved with the positivity constraint, long-object compensation and 125 iterations.

Figure 2.

Numerical simulations on the 3D reconstruction of a 64 × 64 × 64 voxel vesicle model from 71 noisy projections using GENFIRE, EST, FBP and SIRT. (a–c) Three 10-voxel-thick central slices of the vesicle model in the XY, ZX and ZY planes, respectively. The corresponding three reconstructed slices with GENFIRE (d–f), EST (g–i), FBP (j–l), and SIRT (m–o), where the missing wedge axis is along the z-axis. (p) The FSC between the reconstructions and the model, showing that GENFIRE produces a more faithful reconstruction than other algorithms at all spatial frequencies.

Figure 2d,g,j and m show a 10-voxel-thick central slice of the 3D reconstructions in the XY plane using GENFIRE, EST, FBP and SIRT, respectively, where the z-axis is the missing wedge direction. Because there is no missing data in this direction, the reconstructions from all methods exhibit good agreement with the model (Fig. 2a). However, along the missing wedge direction both GENFIRE and EST reconstructions (Fig. 2e,f,h and i) appear to be more isotropic and contain more fine features than FBP and SIRT (Fig. 2k,l,n and o). The Fourier shell correlation² (FSC) between the reconstructions and the model further confirms that the GENFIRE resconstruction is superior at all spatial frequencies compared to other algorithms. This simulation was also performed with no noise and higher noise (Supplementary Figs 2 and 3). In the case of noise-free data with equal slope angles, EST produces slightly better results than GENFIRE as no interpolation is needed in EST. However, in practice this idealized scenario does not occur, and our results show that for even moderate noise levels GENFIRE produces better results. SIRT introduces a form of regularization to the reconstruction, which reduces missing wedge artifacts but also appears to compromise the resolution. By accurately assembling a small fraction of the Fourier grid points and using an iterative algorithm with resolution extension/suppression, GENFIRE is able to simultaneously reduce the effect of noise and retain higher resolution information. This capability will be important as scientists continue to solve important problems by pushing imaging systems to their limits.

Numerical simulations on atomic electron tomography

To quantify the GENFIRE reconstruction of 3D nanostructures at atomic resolution with noise and a missing wedge, we generated a 3D atomic model consisting of a 4.3 nm FePt₃ nanoparticle with a chemically ordered face-centered cubic (L1₂) phase. Using this model, 27 annular dark-field (ADF) projections were computed using multislice simulation⁴⁷ (electron energy: 300 keV, probe size: 0.5 Å, C₃: 0 mm, C₅: 5 mm, probe convergence semi-angle: 30 mrad, and the inner and outer detector angles: 48 mrad and 251 mrad). The multislice calculation accounts for a number of nonlinear effects including dynamical scattering. The angular tilt range is ±70.1° and the pixel size is 0.4 Å. For each tilt angle, a total of 10 frozen phonon configurations were averaged. To simulate the convolution effect resulting from finite probe size and other incoherent effects, each image was convolved with a 2D Gaussian function with σ = 0.51 Å. Poisson-Gaussian noise was then added to the ADF scanning transmission electron microscopy (STEM) projections.

After denoising was applied to the projections⁴⁸, this tilt series was reconstructed with GENFIRE, EST, SIRT and FBP, and the results are shown in Fig. 3. Visually, GENFIRE, EST, and SIRT all demonstrate reduction of reconstruction artifacts, though the difference appears more substantial for GENFIRE and EST (Fig. 3a–h). Both SIRT and FBP suffer from aliasing artifacts that produce what appear to be atoms, but are not actually present in the model, outside of the true boundary of the particle (Fig. 3c,d,g and h). These phantom atoms would prove problematic for atom tracing and refinement in AET. The iterative methods have also successfully recovered missing information as indicated by the presence of Bragg peaks in the missing wedge (magenta arrows in Fig. 3i–k). Determination of 3D atomic coordinates is most accurate when the reconstruction is isotropic, thus it is important for the reconstruction algorithm to be robust to noise and the missing wedge problem. Among the four algorithms, GENFIRE produces the best reconstruction of the 3D atomic structure. Individual atoms are clearly visible in the 3D reconstruction (Fig. 3a and e) even though dynamical scattering effects are present in the 2D projections. We attribute the reduction of the dynamical scattering effects to the measurement of many projections at different sample orientations (i.e. a rotational average) in ADF-STEM tomography¹⁰. A detailed discussion of dynamical scattering on AET can be found in the Supplementary Information in ref. 10.

Figure 3.

Numerical simulations on atomic electron tomography. 1.2-Å-thick central slices of a L1₀ phase FePt nanoparticle in the XY and ZX planes, reconstructed from 27 noisy multislice STEM projections with GENFIRE (a,e), EST (b,f), SIRT (c,g), and FBP (d,h), where the z-axis is the missing wedge direction. The red arrow indicates a Pt atom and the white arrow indicates an Fe atom. Central slices in the ZX plane after applying the Fourier transform to the 3D reconstructions obtained by GENFIRE (i), EST (j), SIRT (k), and FBP (l), showing recovery of the Bragg peaks in the missing wedge direction for GENFIRE, EST and SIRT (magenta arrows). Artifacts due to missing wedge effects such as “ghost atoms” are visible in SIRT and FBP (c,d,g and h), but are not present in EST and GENFIRE (a,b,e and f).

Angular refinement simulations

To demonstrate the improvement made by angular refinement, a simulation was performed using the same 27 ADF-STEM projections from Fig. 3. The orientation angle of each projection was randomly shifted up to ±2°, and a random translational shift of ±1 pixel was applied along the x and y-axes. A preliminary GENFIRE reconstruction was performed and used as input to the refinement loop which was run for a total of 5 iterations with an angular search range of ±3° with 0.2° steps, and with normalized cross-correlation as the error metric. The results of this simulation are shown in Fig. 4. The initial and refined angles were compared with the true ones using a normalized angular distance⁴⁹ (Fig. 4a.), resulting in an improvement from an initial average angular error of 2.1° to a refined value of 1.3°. The reconstruction is improved after angular refinement, shown in Fig. 4b and c. The boundary of the nanoparticle is also better defined, with fewer artifacts around the periphery.

Figure 4.

Angular refinement simulations for the GENFIRE reconstruction of the 27 multislice STEM projections used in Fig. 3. (a) The angular error between correct projection angles and the initial misaligned angles (black dots) and the refined ones after 5 refinement iterations (blue crosses), improving an average angular error from 2.1° to 1.3°. (b,c) 1.2-Å-thick central slices before and after angular refinement, showing some Fe atoms in the lower left region are better resolved and the boundary of the nanoparticle is also better defined.

GENFIRE reconstruction on experimental data of a porous material

To perform a quantitative comparison between GENFIRE and other iterative algorithms on experimental data, we acquired a tomographic tilt series of a Mo₃Si alloy annealed at 1100 °C. Mo-Si and Mo-Si-B alloys are resistant to oxidation and creep and are among potential candidates with high melting temperatures to replace currently widely used Ni-based superalloys^{50, 51}. The experiment was conducted on an FEI TitanX 60-300 in STEM mode equipped with a Gatan high-angle annular dark field detector. The microscope was operated at 200 keV with electron beam current ~40 pA, a convergence semi-angle of 10 mrad, and a camera length of 91 mm. A total of 129 projections were collected with a tilt range from −58° and +70° in 1° increments. After background subtraction, the projections were aligned along the tilt axis direction by cross-correlation and along the perpendicular direction using the center-of-mass method^{8, 9}. Reconstructions were performed with GENFIRE and SIRT. The SIRT reconstruction was computed using Tomo3D⁵². Figure 5a and b show the 13.6-nm-thick central slice of the GENFIRE and SIRT reconstruction of a fragment of the sample, revealing a complex 3D porous structure. Along the 0° direction, both GENFIRE and SIRT produce good reconstructions, although fine features are better resolved by GENFIRE (Fig. 5a and b). However, in the missing wedge direction, GENFIRE exhibits significant improvement over SIRT with sharper boundaries and more distinctive 3D pore structures (Fig. 5c and d). Figure 5e and f show isosurface renderings of the reconstructions, where elongation artifacts due to the missing wedge are clearly visible in the SIRT reconstruction, but are reduced by GENFIRE.

Figure 5.

Comparison of GENFIRE and SIRT reconstructions of a fragment of porous Mo₃Si alloy, annealed at 1100 °C. (a,b) 13.6-nm-thick central slices along 0° direction reconstructed by GENFIRE and SIRT, respectively, where fine features are better resolved in the GENFIRE reconstruction. (c,d) 13.6-nm-thick central slices of the GENFIRE and SIRT reconstructions along the missing wedge direction, where GENFIRE shows significant improvement over SIRT with sharper boundaries and more distinctive 3D pore structures. (e,f) Isosurface renderings of GENFIRE and SIRT reconstructions, where elongation artifacts due to the missing wedge are visible in the SIRT reconstruction, but are reduced by GENFIRE.

GENFIRE reconstruction of a frozen hydrated cell

GENFIRE was also used to reconstruct the 3D structure of a frozen-hydrated marine cyanobacterium in a late stage of infection by cyanophages⁵³. A tilt series of 42 projections ranging from −58° to +65° were acquired on a JEM2200FS electron microscope equipped with a Zernike phase plate and recorded on a 4k × 4k Gatan CCD. The projections were binned by 4 × 4 pixels, resulting in images with approximately 1.8 × 1.8 nm² per pixel. The background was carefully removed from each projection based on the average value in a flat region outside of the cell. A marine cyanobacterium was then cropped out from the surrounding regions by smoothing and thresholding each projection to produce a soft-edged mask. Finally, each projection was aligned and normalized to have the same total sum as the integrated density should be conserved. The tilt series was separately reconstructed with GENFIRE and FBP (Fig. 6). The GENFIRE reconstruction was performed for 100 iterations with a loose cubic support, while the FBP reconstruction was computed using IMOD⁵⁴. Several low-contrast features are visible in the GENFIRE reconstruction that are difficult, if not impossible, to identify with FBP. Of particular interest in this dataset was the interactions between the marine cyanobacterium and cyanophages. Figure 6 shows a slice through the reconstructed volumes capturing the penetration of a cyanophage into the cell membrane during the infection process. This interaction has caused a local depression in the cell membrane, and the shown cross section passes through this depression as well as the viral capsid and appendage (Fig. 6c–j). Based on this geometry the cell membrane should be visible on both sides of the interaction, similar to taking a horizontal cross-section through a U-shape (Fig. 6c and d). Although the top side of the membrane is visible in both reconstructions (magenta arrows), the bottom side is only visible in the GENFIRE reconstruction (yellow arrow). Figure 6i and j show isosurface renderings of the penetration of the cyanophage into the cell membrane, where GENFIRE exhibits higher contrast, less peripheral noise, more easily detectable cell boundaries than FBP.

Figure 6.

3D structure of a frozen-hydrated marine cyanobacterium, capturing the penetration of a cyanophage into the cell membrane. (a,b) 5.4-nm-thick slices of the cell in the XY plane reconstructed by GENFIRE and FBP, respectively, where the Y axis is the tilt axis and the beam direction is along the Z axis. Magnified views of the penetration of a cyanophage for the GENFIRE and FBP reconstructions in the XY (c,d), XZ (e,f), and ZY (g,h) planes, respectively. The top side of the membrane is visible in both reconstructions (magenta arrows), but the bottom side is only visible with GENFIRE (yellow arrow). (i,j) Isosurface renderings of the penetration of the cyanophage to the cell membrane. Overall, GENFIRE exhibits higher contrast, less peripheral noise, more easily detectable cell boundaries than FBP.

Discussion

In this article, we present the mathematical implementation of GENFIRE for high-resolution 3D reconstruction from a limited number of projections with a missing wedge. Both numerical simulation and experimental results of materials science and biological specimens indicate that GENFIRE produces superior 3D reconstruction to several other tomographic algorithms. As a Fourier-based iterative method, GENFIRE first computes a small fraction of Cartesian grid points with high precision from 2D projections using Fourier gridding and oversampling, where proper choice of the oversampling ratio is critical to the precise calculation of the small fraction of grid points. It then iterates between real and reciprocal space using the FFT and its inversion. Positivity and support are enforced in real space, while the grid points calculated from the measured data are applied in reciprocal space. As the Fourier data, positivity and support are all convex constraint sets, GENFIRE belongs to the method of projections onto convex sets, whose convergence has been mathematically proven^{55, 56}. This allows GENFIRE to search for a global solution that is concurrently consistent with the measured data and physical constraints. One of the unique features of Fourier-based iterative algorithms such as GENFIRE is that any changes in real space globally affect all the points in reciprocal space and vice versa. This global correlation between real and reciprocal space makes GENFIRE robust to the missing data and missing wedge. In contrast, ART, SART and SIRT perform all the iterations in real space through local interpolation. When there is a missing wedge, the local interpolation in that region becomes less accurate. This explains why GENFIRE achieves better 3D reconstructions than several other tomographic algorithms. Furthermore, compared to EST that is only applicable to single tilt axis data, GENFIRE can not only work with any tomographic geometry, but also performs faster due to the use of the FFT and its inversion for iteration. It must be cautioned, however, that GENFIRE users must be aware of the physical implications of the conditions under which images are acquired with respect to the linearity of the projection. Although nonlinear effects such as dynamical scattering can be alleviated due to a rotational average in tomography¹⁰, the exact extent to which such nonlinear effects degrade GENFIRE needs to be explored in future work.

Another class of tomographic reconstruction methods based on compressed sensing is presently under rapid development^{57, 58}. Compressed sensing assumes that a physically meaningful structure is usually sparse in some domain. If the sparse domain can be found, the 3D structure can in principle be reconstructed from a small number of 2D projections. Compressed sensing tomography typically incorporates mathematical regularization such as total variation minimization⁵⁹, which requires manual tuning of parameters⁶⁰. This is acceptable in certain applications, where the scope of reconstruction targets is limited enough to permit a specialized set of parameters. However, for general tomographic reconstructions, it is not straightforward to optimize these parameters, especially with the presence of missing data and noise. For example, it would be very challenging, if not impossible, for compressed sensing tomography to reconstruct the 3D distribution of point defects in a crystalline specimen. Conversely, GENFIRE uses very general physical constraints and requires minimum manual tuning of parameters. It has recently been used to determine crystal defects such as grain boundaries, chemical order/disorder, anti-phase boundaries and point defects with unprecendent 3D detail^{10, 13}. Furthermore, GENFIRE can be easily adapted to incorporate mathematical regularization to reconstruct 3D sparse objects from a small number of projections. Looking forward, we expect GENFIRE can be applied to a plethora of imaging modalities to address a wide range of scientific problems.

Software availability

The GENFIRE software package with a graphical user interface is freely available at www.genfire-em.com.

Author Contributions

J.M. directed the research; J.Z., W.C. and J.M. discussed and/or acquired the data; A.P.J., Y.Y. and J.M. discussed and developed the algorithm; A.P.J., Y.Y., A.R., M.G.J., J.Z. Y.H.L., G.M., J.A.R. and J.M. performed the numerical simulation and the reconstruction of the experimental data. A.P.J. and J.M. wrote the manuscript with contributions from all authors.

Competing Interests

The authors declare that they have no competing interests.