Optimal 3D single-molecule localization for superresolution microscopy with aberrations and engineered point spread functions

Abstract

Photo-activation localization microscopy is a far-field superresolution imaging technique based on the localization of single molecules with subdiffraction limit precision. Known under acronyms such as PALM (photo-activated localization microscopy) or STORM (stochastic optical reconstruction microscopy), these techniques achieve superresolution by allowing only a sparse, random set of molecules to emit light at any given time and subsequently localizing each molecule with great precision. Recently, such techniques have been extended to three dimensions, opening up unprecedented possibilities to explore the structure and function of cells. Interestingly, proper engineering of the three-dimensional (3D) point spread function (PSF) through additional optics has been demonstrated to theoretically improve 3D position estimation and ultimately resolution. In this paper, an optimal 3D single-molecule localization estimator is presented in a general framework for noisy, aberrated and/or engineered PSF imaging. To find the position of each molecule, a phase-retrieval enabled maximum-likelihood estimator is implemented. This estimator is shown to be efficient, meaning it reaches the fundamental Cramer–Rao lower bound of x, y, and z localization precision. Experimental application of the phase-retrieval enabled maximum-likelihood estimator using a particular engineered PSF microscope demonstrates unmatched low-photon-count 3D wide-field single-molecule localization performance.

Ongoing superresolution microscopy developments are opening up opportunities to explore subcellular structure and function as well as to visualize materials at the nanoscale (1–7). In photo-activated localization microscopy/stochastic optical reconstruction microscopy (PALM/STORM) imaging, random and sparse sets of emitters are allowed to emit independently in time instead of simultaneously, as in normal fluorescence imaging. An image is then formed by the ensemble of precise localization measurements. In this way, the image is not bound by the traditional limit of resolution set by diffraction but rather by the localization precision of the individual molecules. In comparison with other superresolution microscopy techniques (4–6) PALM/STORM is attractive because it enables the recording of multiple single molecules in parallel, with each molecule experiencing limited photo-activation cycles. The technique is also appealing for simultaneously tracking individual molecules over time or measuring the distance between pairs of molecules (8). There is a significant current interest in decreasing the data acquisition time by engineering new fluorescent proteins and organic fluorophores (9) along with optical improvements in photon collection (10, 11). Significant efforts are also underway to reduce systematic errors such as vibrations, drift, and imperfections of the detector (8), as well as bias from asymmetric molecular emission (12). However, existing methods to improve 3D localization use custom estimation algorithms that are not efficient in terms of achieving the fundamental localization limits (13–19).

Therefore the newly discovered frontiers of optical resolution and functionality—such as quantitative tracking and distance measurement—have not been fully realized yet. Optimal 3D localization procedures are critical to increase resolution for a given number of detected photons and to reduce data acquisition time for a given target resolution. Another practical problem is optical aberrations that distort the single-molecule images and negatively affect the localization precision. Accordingly, this paper addresses the problem of optimal 3D localization for arbitrary 3D point spread functions (PSFs) in the presence of noise while providing a solution that rigorously accounts for the inevitable optical system aberrations.

Current Optical Methods for 3D Single-Molecule Localization

Four main techniques are prevalent in far-field 3D single-molecule localization microscopy: biplane detection (13), astigmatic imaging (14), double-helix (DH) PSF (Supporting Information provides an introduction to this topic) (11, 15, 16, 20), and interferometric PALM (10). The first three are of particular interest because they require minor modifications of a standard microscope and operate over relatively long depth ranges (up to 2–3 μm for DH-PSF). In contrast, interferometric PALM offers superb axial resolution but requires more complicated interferometric arrangements and operates over a 200 nm range (10, 21).

To evaluate the performance of each system, the fundamental limit to 3D localization precision can be quantified by means of the Cramer–Rao lower bound (CRLB) (16, 22, 23) (see Supporting Information for an introduction to this topic). The CRLB also serves as a benchmark for estimator implementations, by which the realized precision of the estimator should approach this fundamental limit. These uses of the CRLB are increasingly relevant in single-molecule imaging, where the number of photons is inherently limited. Estimators based upon simple geometrical interpretations of the PSF [i.e., centroid (14), rotation angle (11, 15, 16), 2D Gaussian-fit (13–15, 17–19)] have been implemented and are computationally fast but not optimal because they do not reach the fundamental limit of precision for a given physical system. These methods are suboptimal both because of the coarse approximation of the PSF spatial profile and also because of the lack of specific noise statistics.

The inclusion of explicit noise statistics into an estimation algorithm enables the definition of a most-likely 3D position, i.e., a maximum-likelihood estimator (MLE) (24–27). Prior demonstrations of 3D MLEs have only included optical systems that can be described with analytical solutions. This represents a limitation for the realizable optical design space that excludes the benefit of numerically optimized solutions. We point to the CRLB analysis demonstrating the 3D performance advantages of one such example, the DH-PSF, relative to the analytical designs as justification for the necessity to include optimized PSFs (22). In addition, existing MLEs do not explicitly address a method for including inevitable experimental aberrations in the optical system. Inclusion of aberrations is critical in 3D methods because microscopes are in principle corrected for aberrations only across a single transverse plane.

Therefore no method has been demonstrated that solves the problems of efficient 3D estimation for generalized optical systems or accurately account for the effects of aberrations induced by both the sample and object. In light of this, here we propose and demonstrate an information-efficient algorithm to achieve the CRLB for 3D localization in the presence of aberrations and using arbitrary PSFs.

3D Phase Retrieval (PR) for Optimal Estimation

Optimal estimation requires a faithful model that can be matched to experimental calibration. Unfortunately, a detailed and exhaustive calibration of the PSF at single nanometer shifts, if possible, is not always practical because of the required large data acquisition sets (19, 28), which either become less precise or demand longer calibration times. Hence, we introduce a PR method (29, 30) to accurately recover the 3D PSF at arbitrary coordinates over a continuous 3D domain based on only a few calibration measurements. PR has been used in microscopy to correct aberrations in classical fluorescent imaging using a single-pupil function model (30). This model is an approximation equivalent to a depth-invariant PSF that leads to residual errors even when the vector nature of the fields, the finite size of beads, and noise are taken into account (30). Whereas this approach has been shown to provide good results in conventional (diffraction limited) deconvolution microscopy, its effectiveness is not guaranteed for engineered PSFs and superresolution imaging, where dimensions are one order of magnitude smaller (See Supporting Information). Therefore, the PR method proposed here assumes a local, depth-variant model, and is shown below to be effective both for engineered PSFs and for PALM/STORM imaging to improve localization precision and ultimately resolution.

To generate a proper model of the imaging system that accounts for the presence of aberrations and the arbitrary optical system design, a set of calibration images, {I_z1,I_z2,…,I_zN}, and a PR algorithm are used to solve for the complex field of the PSF,

[1]

with zn = z1…zN indicating N axial positions. A diagram of the PR algorithm demonstrating a working example for estimating the complex field at an arbitrary plane z₀ is provided in Fig. 1. For any location of the molecule in space, an accurate model of the PSF is obtained from the images |f|² by continuously propagating along Δz and shifting in Δx, Δy according to the wave equation that governs the evolution of the PSF,

[2]

where M is the magnification, λ is the wavelength, and f_4F is the focal length of the lens on the image side (see also Fig. 2). The PR algorithm can be implemented using a minimum of two axial positions for calibration images of the PSF, taken at well-known coordinates where n = 1,2. However, increasing the number of calibration images for PR has been documented to improve convergence and robustness in the presence of noise (31). Prior methods have assumed that these field estimates can be averaged across the entire depth of focus to provide a best estimate for the system pupil, but this assumption is overly restrictive for high numerical aperture systems as shown in ref. 30, leading to residual errors in the estimated PSFs. Although these errors can be tolerated in applications such as classical diffraction limited deconvolution imaging, without further modifications they could introduce significant errors in localization-based superresolution microscopy (see Supporting Information). Therefore, a method that departs from the single-pupil approximation, which implicitly assumes a depth-invariant PSF, is required. It is with this in mind that we implement a depth-variant method, using the PR algorithm with three local and contiguous axial calibration images to find the complex-field at each of the N depth positions. Hence, in this report we assume each estimate to be the best local representation of the associated field and retain this for later use in the MLE.

Fig. 1.

The phase retrieval algorithm for PSF recovery and interpolation. (A) A sequence of three local calibration images (I_z0-Δz, I_z0, I_z0+Δz) is used for recovering the local depth-variant complex amplitude PSF of the optical system that is associated with the aberrated and/or engineered PSF. The search procedure requires five steps (S1–S5) per iteration until a solution converges at plane z₀, where the local phase is shown below the intensity image. Comparison of the axial dependence for the experimental and interpolated images of the aberrated DH-PSF are given in panels (B) and (C), respectively. (B) Projections of the experimental images along the x and y axes with axial propagation shows the discrete character of the calibration data. (C) The results from the PR algorithm behave smoothly—indicative of proper interpolation of the expected PSF at intermediate image planes to be used for MLE.

Knowledge of the absolute value of the amplitude for f_z0-Δz, f_z0, and f_z0+Δz is provided from the calibration images as , , and , respectively, and is used to find a unique estimate of f_z0. Referring to Fig. 1A, the propagation of -Δz in step S1 is described by a defocus operation to calculate f_z0-Δz, i.e., a shift in object position of (see Eq. 2). After this numerical propagation, the amplitude constraints of are enforced and the phase component of f_z0-Δz is kept in step S2 as this represents an improvement in the estimation of this function. This new estimate is then forward-propagated to provide an updated estimate of f_z0 as step S3. The forward-propagation is continued to z0 + Δz, the constraints again enforced as step S4, followed by a subsequent back-propagation to the starting point z0 to complete the algorithm with step S5. Steps S1–S5 represent a single iteration of the PR algorithm. Iteration continues until a measure of convergence such as the sum-squared-error shows asymptotic behavior. Note that in practice, it has been documented that an optimum Δz exists for a given noise level of the calibration images (32). In our experiments, an average of 27 images are taken at each axial position to decrease the noise present in the calibration image (where each axial position is taken at 100 nm intervals). The PR can be further refined—if necessary—using known methods that account for undersampling and broadband emission (33). A generalization of the method presented here is also possible for nonisoplanatic systems.

The result of the PR is a rigorous interpolation of the calibration data that can be used in the MLE process. Note that this process has incorporated the entire optical system into the calibration process without making any significant assumptions about amplitude/phase modulation devices or aberrations. Therefore, optical PSF design and systematic aberrations can be incorporated seamlessly for optimal estimation.

3D Maximum-Likelihood Localization Estimation

The optical system description given by the experimental PSF and, if needed obtained via the PR process, is utilized in the second component of this method. Given an experimental image of a single-molecule I_exp, this optimal estimator searches the object space for the most-likely PSF match using Eq. 2, i.e., the position of the molecule that maximizes the likelihood function,

[3]

Here, is the estimate resulting from the maximization and is the set of all 3D positions; is a function representing the likelihood of the match at in the presence of noise processes such as Poisson or Gaussian (the Gaussian case is treated in Supporting Information).

A proper initial guess accelerates the convergence and avoids local maxima for the optimization required to solve Eq. 3. Accordingly, the starting axial location Δz⁽¹⁾, with (1) denoting the first iteration of the search, is found using the calibration image that best matches the experimental image by minimizing the sum-squared-error of the Fourier coefficient amplitudes (See Supporting Information). This results in an initial z estimate that is unrelated to the transverse position of the molecule. The transverse coordinates of the initial estimate (Δx,Δy)⁽¹⁾, are found separately using a cross-correlation of the experimental image with the previous (best match) calibration image. The maximization in Eq. 3 is then achieved using the iterative update solution (24),

[4]

to find the most-likely match, where j∈{Δx,Δy,Δz}, p is the iteration number and γ is a gain coefficient on the update (expressions for evaluating the derivatives are given in the Supporting Information).

Performance with Simulation Data

The estimation efficiency of this localization method is quantified by simulating a DH-PSF microscope with and M = 91x, imaging a single molecule with average isotropic emission at λ = 515 nm. The optical configuration is designed to match that of our typical experimental implementation, as given in Fig. 2 (see also Supporting Information). The behavior of the DH-PSF image can be approximately described as a pair of intense lobes that rotate about the mutual center of mass as the object translates axially, shown both in the experimental and PR data of Fig. 1 B and C, respectively, as well as Fig. 2.

Fig. 2.

Optical layout of PSF engineered microscope. The DH microscope collects the light emitted from the fluorescent sample. A traditional microscope with a 1.3 NA objective is appended with a 4F optical setup, used to reimage the intermediate image plane onto the detector. A spatial light modulator (SLM) is placed in the Fourier plane of the 4F setup and is used to code the optical pupil of the microscope. When the DH-PSF phase mask is present on the SLM, the light emission from the sample is encoded for subsequent imaging onto an electron-multiplying charge-coupled device, located after the second Fourier lens (Lens 2). Experimental images are given in the right pane as a nanoparticle is translated axially through focus.

The PR algorithm uses the average of 100 noisy image realizations, taken at three axial positions (z = -100 nm, 0 nm, and +100 nm) to form the calibration image set. Note that the PR algorithm can be run initially for system characterization and then store the recovered optical system pupil function for later use to speed up computation time.

The PR-MLE performance is tracked as the number of photons detected from the emitter is varied across a representative range. Poisson noise is considered the dominant noise process, consistent with ideal experimental conditions, with a likelihood function given by,

[5]

where the image size is M × N, (m,n) are indices that identify each pixel, and b is a constant background noise level. To increase the per-pixel signal-to-noise ratio (SNR), the DH-PSF was sampled at one-half the Nyquist sampling rate—consistent with our experimental conditions*. To isolate the performance under the shot-noise limit, no background noise is included in the simulation results presented here. Background noise is included in the experimental results of the next section.

In Fig. 3, the MLE performance is compared for two likely scenarios along with the associated system CRLB (see Supporting Information for further discussion of CRLB). The first scenario characterizes the performance of the MLE algorithm when the optical pupil function was known a priori. This establishes the performance limit without introducing any potential errors from the PR algorithm. The second scenario characterizes the PR-MLE method using the PR results from the calibration images to obtain the pupil function and 3D complex-valued PSF. About 20 iterations are necessary for the MLE to converge on each estimate. We provide a measure of the estimator efficiency as —the ratio of the estimator precision (measurement standard deviation) to that of the CRLB. To develop a sense for the performance across a broad range of possible conditions, we average the efficiency of the estimator over a photon count range of 300–1,100 photons. For a system known a priori, the resulting average efficiency is ε_MLE,ap(x,y,z) = (1.0 ± 0.1,1.0 ± 0.1,1.1 ± 0.1) along the x, y, and z directions^†. Therefore, these results demonstrate that the performance of the MLE is efficient. For the PR-MLE the efficiency along the x, y, and z directions is ε_MLE,PR(x,y,z) = (1.0 ± 0.1,1.5 ± 0.1,1.1 ± 0.1). The minor loss of precision for the PR-MLE along the y coordinate is due to both the sub-Nyquist sampling conditions and the presence of noise in the calibration images used for the PR estimate. The results of Fig. 3 imply that the PR-MLE estimator is efficient in reaching the performance limit in the shot-noise-limited case (the Gaussian case including aberrations is treated in Supporting Information, also showing efficient estimation)

Fig. 3.

Localization efficiency tests as measured by applying the MLE to 100 simulated, noisy images and varying the number of photon counts. The lines indicate the theoretical lower bound (CRLB) of the precision for which the x, y, and z-axis positions may be localized in the shot-noise limit with an emitter in focus (z = 0). The bound increases rapidly as the number of photons collected in the image decreases. The colored circles indicate the associated maximum-likelihood (ML) estimation precision when the pupil function of the optical system is well known and no PR is necessary. The colored triangles indicate the ML precision when the phase-retrieval algorithm is implemented.

Performance with Experimental Data

A sample of PtK1 (rat kangaroo kidney epithelial) cells expressing PA-GFP-tubulin (a photo-activatable green fluorescent protein) is examined experimentally using the DH microscope of Fig. 2 in the PALM/STORM modality (see Supporting Information). The axial calibration data is shown in Fig. 1B and the PR results are given in Fig. 1C. A total of 144 individual single molecules are identified in this sample under the constraint that each is present in four sequential images and yielding 998 total localization measurements. An average of 2,087 ± 291 photons per image was collected for this ensemble. The molecules were found to be dispersed throughout a 1.9 μm axial depth and a 25 × 25 μm field of view (see Supporting Information). The histograms in Fig. 4 show estimation results of the PR-MLE on the data-set of single-molecule measurements, each position shifted to have a mean located at the origin. Significant background is present in the experiment (an average of 349 ± 57 photon counts per pixel; see Supporting Information for background subtraction and estimation procedures). The PR-MLE estimation precision (standard deviation) on this collection of molecules along the x, y, and z directions is σ_MLE(x,y,z) = (17,10,19) nm.

Fig. 4.

Localization and photon counting results from 998 estimations of single-molecule positions in a PALM experiment with a DH-PSF. (A) The 3D scatter plot of these measurements demonstrates localization within a 17 nm × 10 nm × 19 nm volume (standard deviation). (B) Typical experimental image of one PA-GFP molecule (2,140 photons detected). (C) The distribution of counts per image reveals a normal distribution with a mean number of 2,087 photons. (D–F) The individual histograms of the x, y, and z-axis estimation from the ensemble used in panel A.

A sense for the experimental estimator efficiency of the PR-MLE is developed by comparison of the reported MLE precision with a representative CRLB by assuming average emitter characteristics found from the dataset (i.e., positioned at the mean axial position, emitting the average number of photons with the average background level). This CRLB is found to be σ_CRLB(x,y,z) = (12,16,23) nm. The average estimator efficiency of all three position estimates is therefore 0.95, indicating that, on average, the estimator is indeed efficient at localizing experimental emitters within a volume. Note that this is only an estimate of the performance limitations for the experimental system and that the nonlinear relationship with background count levels (approximately 16% measured variation), assumptions of a uniform background, the changes of the signal level from the emitter (approximately 14% measured ), as well as the averaging of performance across an extended range of axial depth, all contribute to real deviations from this CRLB estimate, which in this case, should only be taken as indicative of the expected performance of an efficient estimator. During the experiments no significant anisotropic effects were noticed due to the rotational freedom of the fluorescent proteins. If present, these anisotropies could lead to systematic errors that need to be corrected.

To illustrate the applicability of the methods presented above, we have performed distance measurement, tracking, and PALM experiments. For the distance measurement demonstration, two molecules were identified in the PALM experiment and their 3D distance measured as shown in Fig. 5. The first molecule (right-most) is localized to σ_SM1(x,y,z) = (17,16,25) nm and the second molecule (left-most) is localized to σ_SM2(x,y,z) = (9,19,20) nm. After propagating the random error in localization of each molecule, the distance separating them was determined as 778 ± 22 nm. The experimental data points are given as solid dots and each molecule position is visualized as an ellipsoid where the semiaxes are given by the respective standard deviation of each localization measurement. Tracking and PALM experiments are reported in the Supporting Information.

Fig. 5.

Measurement of the distance between two PA-GFP molecules. Nine total measurements are used to localize the individual molecules. Because each molecule is well localized a high precision distance measurement can be calculated.

Discussion

The experimental single-molecule localization performance compares favorably with recent 3D localization experiments in PALM/STORM. A rigorous comparison of the various experimental techniques is difficult because the available reports relate to different samples and fluorophores (10, 11, 13–15). Furthermore, 3D PALM/STORM reports do not always provide complete data (3D precision, number of detected photons, background noise level, axial range, etc.). Besides, many studies use isolated beads rather than single molecules to characterize the system’s performance, which, although useful, provide favorable detection conditions with no background noise from other emitters, no auto-fluorescence, and limited aberrations. One way around this problem is to consider fundamental assessments of the performance limits, which have recently shown the advantages of engineered PSFs (22). Another informative approach is to compare actual experimental single-molecule performances based on quantitative measures of resolution and axial range as explained in what follows.

For the shot-noise limiting case, neglecting for the moment background and pixelation noise, the localization precision follows the scaling law (23, 34). Hence, one can use the photon-normalized geometric mean precision, , as a resolution metric to compare 3D systems. According to this metric the system demonstrated here achieves about 50% better precision than prior 3D PALM/STORM demonstrations using astigmatic (14) and biplane (13) imaging (corresponding to approximately 3 times smaller minimum resolvable volume) with more than twice the depth range. It thus becomes closer to achieving the axial resolution of interferometric systems (10) while providing one order of magnitude longer axial range with a much simpler system (see Supporting Information for comparison details). It should be noted that the current DH-PSF experiment was performed with fluorescent proteins and over a long depth range, both factors that negatively affect the signal-to-background noise ratio.

The Supporting Information presents an experimental validation and theoretical analysis of the accuracy (bias and systematic errors) of the method, showing that with proper calibration, the procedures are faithful to represent the actual emitter positions and distances among them.

These results show that the integration of PSF engineering, photo-activation-localization imaging, and optimal estimation enables one order of magnitude improvement in resolution along each spatial direction when compared with classical fluorescence imaging. This translates into three orders of magnitude improvement in the resolvable volume over a wide 3D field of view. In addition, the unknown, systematic aberrations may be completely characterized through comparison of the PR recovered pupil and that of the expected pupil. This data may then be used to provide the user additional information concerning the sources of the aberration (e.g., spherical aberrations due to the distance from cover slip), representing a useful by-product of the method.

Conclusion

In conclusion, the optimal estimator based on a (PR-enabled) MLE, tailored to match the conditions found in single-molecule imaging, is efficient in reaching the 3D localization performance limits with arbitrary PSFs. The rigorous depth-variant PR interpolation of calibration data takes into account systematic errors caused by unknown and unavoidable 3D optical aberrations. The optimal 3D localization algorithm in conjunction with a DH-PSF shows the best 3D localization precision in PALM/STORM systems with an extended depth range, enabling an approximately 1,000-fold improvement in resolvable volume with respect to classical fluorescence microscopy.