A Theoretical High-Density Nanoscopy Study Leads to the Design of UNLOC, a Parameter-free Algorithm

Abstract

Single-molecule localization microscopy (SMLM) enables the production of high-resolution images by imaging spatially isolated fluorescent particles. Although challenging, the result of SMLM analysis lists the position of individual molecules, leading to a valuable quantification of the stoichiometry and spatial organization of molecular actors. Both the signal/noise ratio and the density (D_frame), i.e., the number of fluorescent particles per μm² per frame, have previously been identified as determining factors for reaching a given SMLM precision. Establishing a comprehensive theoretical study relying on these two parameters is therefore of central interest to delineate the achievable limits for accurate SMLM observations. Our study reports that in absence of prior knowledge of the signal intensity α, the density effect on particle localization is more prominent than that anticipated from theoretical studies performed at known α. A first limit appears when, under a low-density hypothesis (i.e., one-Gaussian fitting hypothesis), any fluorescent particle distant by less than ∼600 nm from the particle of interest biases its localization. In fact, all particles should be accounted for, even those dimly fluorescent, to ascertain unbiased localization of any surrounding particles. Moreover, even under a high-density hypothesis (i.e., multi-Gaussian fitting hypothesis), a second limit arises because of the impossible distinction of particles located too closely. An increase in D_frame is thus likely to deteriorate the localization precision, the image reconstruction, and more generally the quantification accuracy. Our study firstly provides a density-signal/noise ratio space diagram for use as a guide in data recording toward reaching an achievable SMLM resolution. Additionally, it leads to the identification of the essential requirements for implementing UNLOC, a parameter-free and fast computing algorithm approaching the Cramér-Rao bound for particles at high-density per frame and without any prior knowledge of their intensity. UNLOC is available as an ImageJ plugin.

Introduction

Cellular processes rely on the proper stoichiometry and coordinated spatial organization of molecular actors. In this regard, single-molecule localization microscopy (SMLM) provides reliable information on molecular distributions, interactions, and biological processes by counting and localizing single fluorescent particles (1, 2). As such, by aiming to list the position of single molecules (3, 4, 5), SMLM differs significantly from super-resolution microscopy techniques, which generate reconstructed images based on density distribution of the fluorescent signals (6, 7).

Although SMLM imaging is, technically speaking, not overly complex, ensuring the robustness of the super-resolution observations requires that high levels of attention are given to rigorous sample preparation, image recording, and data analysis (2, 8, 9). SMLM imaging is best designed for describing the molecular organization in biological samples that have been fixed to overcome impaired localization precision in measurements of dynamic processes in living cells due to possible fast molecular diffusion occurring during the camera exposure time (10). To a lesser extent, and for the same reason, insufficient sample fixation does alter the localization precision and consequently the final resolution of the reconstructed images (8, 11). SMLM imaging also requires the design of well-defined fluorescent labeling probes to preserve the localization precision (10, 12). This implies ascertaining probe specificity, minimizing the steric constraint, or optimizing the dark/fluorescent state ratio (13). Finally, the quality of SMLM imaging strongly relates to robust data processing (8, 9). If performed optimally, however, SMLM does offer improved visualization and quantification possibilities to explore cellular functions. It renders possible analysis of the organization of molecular complexes through the visualization of their individual molecules rather than through inference based on results of an averaged measure of an ensemble (14, 15).

In essence, the theoretical principle governing accurate localization was based on the imaging of isolated particles (16, 17, 18, 19), and SMLM analyses were initially performed with algorithms fitting the point spread function (PSF) of such isolated particles (20). Consequently, the impact of the signal/noise ratio (SNR), but also the density D_frame, i.e., the number of fluorescent particles per μm² per frame, have since been examined to delineate the expected precision of SMLM data (21, 22, 23, 24, 25, 26). Initially, accurate localization with a one-Gaussian fitting hypothesis requires effective low-density (LD) imaging conditions (Fig. 1 a, left panel). However, such conditions intrinsically undergo frequent experimental exceptions. Indeed, despite low density per frame, high local densities are possible because of a biological distribution of the particles or the intrinsic stochastic nature of SMLM methods. Moreover, complex variations of the background are highly frequent in terms of both space and time. Altogether, this fully justifies the quantification of SMLM data with an accurate high-density (HD) mode of analysis.

Figure 1.

Enumeration, precision, and bias for single-molecule imaging. (a) Scenes with one (effective LD conditions) or two particles (HD and NR conditions) fitted with a one- or two-Gaussian hypothesis and the incidence of the enumeration and localization precision ± bias are shown. (b) An NR scene analyzed using a likelihood ratio test (PD ≈ 0 at PFA = 10⁻⁴ and PD = 0.2 at PFA = 10⁻²) is shown, with indistinct residues between the two fitting hypotheses (see Supporting Materials and Methods, Note S3).

A scene with two particles recapitulates the general problem (Fig. 1 a, middle panel). Postulating LD conditions on these data generates not only a lack of precision but also a bias regarding their localizations. The retrieval of accurate precision and unbiased localization data is possible by way of a two-Gaussian fit under an HD hypothesis. When the two particles are too close to each other (Fig. 1 a, right panel), the scene becomes nonresolvable (NR). The HD hypothesis no longer works, as shown by indistinguishable fitting residues (Fig. 1 b). Thus, an accurate enumeration and localization of particles relies not only on the SNR but also on D_frame. Other analytical approaches have been developed for HD SMLM data, such as those based on compressed sensing (27, 28), Bayesian estimation (29), or deconvolution (30) analyses. Although they do provide the local density of the emitting particles, none reaches the single-molecule localization precision obtained by the multi-Gaussian fitting methods (24, 25, 26) (see also (31) for review).

A theoretical defining of the reliability of SMLM observations is a prerequisite to assessing accuracy and robustness for localization precision. By simulating realistic conditions, we investigated localization precision on a theoretical two-particle scene. Previous studies have determined the bias on the localization precision based on the Cramér-Rao bound (CRB) (32, 33) determined at known intensity (21, 22, 23, 24, 25, 26). Here, we characterize this bias based on an appropriate estimator and the CRB without prior knowledge of the intensity to avoid a noticeable overestimation on the precision of localization for data with high local density/frame or complex background or both. Our theoretical study prompted us to develop tools for accurate quantitative SMLM analyses: 1) a unique density-SNR space diagram that enables standardized evaluation of the localization accuracy expected from experimental data, and 2) UNLOC, a real parameter-free and fast computing algorithm available as a plugin for ImageJ (34) and especially useful for any inexperienced researchers requiring rigorous SMLM quantification.

Materials and Methods

Theoretical framework considerations

The theoretical statistical study was based on a general signal-dependent noise model of the intensity x_p at pixel p = (i, j) as a proper model matching the stochastic processes that occur during experimental acquisition of SMLM (Eq. S1; Supporting Materials and Methods, Note S1). For the sake of generalization, we expressed the inter-particle distance d by the characteristic dimension r₀ of the PSF and the SNR in a logarithmic decibel scale (SNR_dB = 10log₁₀SNR). All derivations and analyses are detailed in Supporting Materials and Methods, Notes S1–S5. Acronyms and symbols are recapitulated in Table S1.

PSF size

We hypothesized that the PSF exhibits a classical integrated Gaussian profile with a characteristic size r₀ represented by the full width at half maximum (FWHM) of the peak according to the following:

$$\text{FWHM}=2r_0{\left(2\ln 2\right)}^{1/2}\approx 2.35r_0.$$ (1)

For generalization, the distances are expressed in r₀ units, which approximated to ∼130 nm for an optical pixel size of 107 nm (r₀ = 1.25 pixels) as determined on our microscope. At this characteristic size, 86% of a single signal power was within 3 × 3 pixels, with the power defined as ${\sum }_p{\left(\alpha g_p\right)}^2$, with αg_p the signal intensity at the pixel p.

Definition of the SNR as a single contrast parameter

A pertinent contrast parameter should describe the difficulty of the task, i.e., the expected precision, independently of the data to ascertain the same precision of particle localization for data acquired on different equipment. We chose the SNR as an efficient contrast parameter, which was calculated using the following equation, assuming that $\iint g^2\left(x,y\right)dxdy=1$:

$$\text{SNR}={\alpha }^2/{\sigma }_{m_0}^2=\left(1/4\pi r_0^2\right)\left(I^2/{\sigma }_{m_0}^2\right)=\pi r_0^2\left(I_{peak}^2/{\sigma }_{m_0}^2\right),$$ (2)

where α is the amplitude of the signal, ${\sigma }_{m_0}^2$ the variance of the fluorescent background is m₀, I is the total intensity of a particle (in number of counts), I_peak is the maximal peak intensity, and r₀ is expressed in pixel units. As compared to the classical contrast parameters defined by the sole signal intensity or by the ratio of the particle signal to the background level, the SNR alone recapitulates the expected achievable precision (Fig. S1). The performance limits are fixed by the CRB (32, 33), which, for a given density D_frame, depends on (SNR, m₀, G) but is only recapitulated by the SNR:

$$\text{CRB}\left(\text{SNR },m_0,G;D_{frame}\right)\approx f\left(\text{SNR};D_{frame}\right),$$ (3)

where $G={\sigma }_p^2/2m_p$ is the electron-multiplying gain, which differs from the electron-multiplying charge-coupled device (EMCCD) gain sets experimentally.

Signal-dependent noise model

An EMCCD or complementary metal-oxide semiconductor camera has been designed for low-light imaging. However, experimental SMLM image acquisition tends to deviate significantly from these conditions and covers a wide range of intensities, from a few hundred to thousands of photons in the background and signal areas (Supporting Materials and Methods, Note S1). Because the PSF corresponds to the average distribution of photons for spatially isolated punctate signals, the particle intensity contributes toward the SNR and the quality of the detected peak shape. Precise particle localization implies that an appropriate SNR has been achieved and, hence, a sufficient photon flux recorded during the integration time.

We considered a general signal-dependent noise model of intensity x_p at pixel p = (i, j) as a realistic model of the signal fluctuations that approximates these conditions:

$$x_p=\mathcal{N}\left(m_p,{\sigma }_p^2=2G{m}_p\right),$$ (4)

where m_p is the average intensity for pixel p, and σ_p is its standard deviation. This equation has been corroborated by others (35).

Choice of the MMSE as appropriate estimator

The use of a minimal mean square error (MMSE) estimator has received increasing interest (20, 36, 37). Under LD conditions, this estimator corresponds to a simple filtering operation (Supporting Materials and Methods, Note S2). Under HD conditions, the maximal likelihood estimator (MLE) and MMSE estimator show a similar mean square error that is similar to the CRB, as well as having very limited bias in cases in which two particles P₁ and P₂ are present at variable distances or at variable SNRs but at a constant SNR difference (Δ_SNR) (Fig. S2). However, the MLE shows more bias than the MMSE estimator upon incorrect detection of the number of particles or for images with a nonhomogeneous background (Figs. S3 a and S8 b; Supporting Materials and Methods, Note S5). Moreover, an MMSE estimator remains less complex than an MLE, which requires optimization of Eq. S25 and, consequently, a higher computational cost.

Generation of synthetic data

All synthetic data were generated with MATLAB (The MathWorks, Natick, MA). Images had to meet the conditions of a signal-dependent noise model, as defined in Eq. 4, to achieve a controlled scene. The PSF of unitary power was defined by three parameters, SNR, r₀, and w_n, where w_n is the size of the window support. Unless otherwise specified, the synthetic data were generated based on the following values: r₀ = 1.25 pixels, m₀ = 300, G = 1, pixel size = 107 nm, and SNR_dB = 25 decibels (dB).

UNLOC algorithm

Description

UNLOC provides a list of coordinates and associated parameters for each detected particle for a posteriori quantification and image reconstruction. The algorithm is based on the decision theory, which only requires the probability of false alarm (PFA) value to be set without the initialization of any parameters relative to the data (SNR, particle density, or background level). UNLOC uses an iterative up-and-down strategy (38) that alternates an overestimation of the number of particles to obtain a minimal fitting residue with a generalized likelihood ratio test (GLRT) to delete useless particles. UNLOC calculates the position errors based on the CRB for particles at high density per frame and without any prior knowledge of their intensity. The UNLOC principle is illustrated in Fig. S9, and the mathematical details are provided in Supporting Materials and Methods, Note S6.

Performance evaluation

We evaluated UNLOC performance in terms of 1) localization accuracy, 2) inhomogeneous density distribution, and 3) computation time.

Localization accuracy on synthetic data

A first chart assessed the limits for a simple case with two particles at different SNRs (from 20 to 30 dB) and variable interparticle distances (from 0.5 to 6 r₀) (Fig. S11); the alphanumeric characters and r₀ scales were generated at high SNR and under LD conditions to avoid any interference with the evaluated signals. A second chart reproduced the density-SNR space diagram with the alphanumeric characters defined by an SNR and local density ranging from 20 to 40 dB and 0.1–5 part/μm²/frame, respectively (Fig. S13). Briefly, each character was designed by dots in a 6 × 12 subpixel area with a subpixel size of 0.1 pixel. The local density was mimicked by the illumination rate of subpixels: the more dots of a given character that were illuminated simultaneously, the higher the local density.

Inhomogeneous density distribution

Synthetic data mimicking clustered molecules by varying cluster and particle densities (3–10 clusters/μm² and 0.1–1 part/μm²/frame) were generated. Clusters of ∼97 particles with a 100 nm diameter at 27 dB of SNR were randomly dispersed in a 10 × 10 μm image (i.e., 94 × 94 pixels), resulting in a variable total number of particles. The UNLOC results and ground truth data were analyzed using two clustering algorithms: a standard clustering algorithm, DBSCAN (39), and a dedicated algorithm for SMLM data, SR-Tesseler (40) (Table S2).

Computation time

All analyses on synthetic and experimental data were computed on the Windows operating system using a Dell Precision T7910 (Dell Inc., Round Rock, TX) with a 2 × 12 cores Intel Xeon processor (E5-2687W v4, 3 GHz) and 128 GB of RAM. Note that, using the same computer, the computation time is reduced by ∼10% on the Linux operating system.

Software package

The UNLOC ImageJ plugin, including a user’s manual and the scripts for the charts, is freely available for academic and nonprofit use as a supporting software package at http://ciml-e12.univ-mrs.fr/App.Net/mtt/ (including updated versions). The content of the software package is detailed in the Supporting Material.

Sample preparation, data acquisition, and analysis

Alexa Fluor 647-phalloidin labeled actin filaments in fixed COS-7 cells (ATCC CRL 1651; ATCC, Manassas, VA) were prepared as described in (41). Quantitative experiments were performed on DNA-origami with GATTA-PAINT HiRes 80R and 20R nanorulers (GATTAquant, Hiltpoltsein, Germany).

All acquisitions were made using total internal reflection fluorescence illumination on a custom-built microscope, as described in Fig. S15 and Table S3, with a 647 nm laser at 4 kW.cm⁻², an axial drift correction by the autofocus module, an EMCCD gain set at 100, and ∼57,000 frames at 36 ms/frame and ∼25,000 frames at 120 ms/frame for cell and nanorulers, respectively.

Unless otherwise stated, data were analyzed with UNLOC in HD mode with a low spatial frequency variation of background, a reconnection process with one off-state lifetime frame, and an integrated Gaussian rendering process after drift correction by correlation. None of the results were filtered except those from the nanoruler analyses, for which a localization precision filter was set to reject particles with σ_xy > 0.05 pixels ∼ 7.6 nm for comparison with the ThunderSTORM analyses. Moreover, ThunderSTORM was tuned to reach the best performances by using the “multi-emitter fitting” mode, an image filtering set by “difference of Gaussian” for synthetic data or “wavelet” for experimental data, and a “local maximum method” for the localization of molecules with the “weighted least squares” fitting method. The UNLOC and ThunderSTORM results were evaluated using GATTAnalysis v1.5 software (GATTAquant).

The Fourier ring correlation (42) was plotted to measure the local resolution in a heterogeneous image (Fig. S14 b).

Results and Discussion

Theoretical statistical study

The SMLM analytical methods used can be divided into two categories. The first one primarily aims at determining the localization of isolated particles (low-density data) (16, 17, 18, 19, 43). As long as the data comply with low-density at constant background, methods within this first category nearly achieve the best accuracy achievable (CRB) by an unbiased estimator (MLE) of the signal parameters. The second category is specifically designed for high-density data sets or fast computing constraints; the images are reconstructed without necessarily achieving the best detection performance or localization precision (27, 28, 29, 30, 44). Combining the benefits of these two categories of analytical methods should maintain fast processing for unbiased particle localizations by approaching the CRB under various SNR and D_frame conditions.

We defined the SNR as a pertinent contrast parameter (Fig. S1; Supporting Materials and Methods, Note S1) and D_frame as the number of fluorescent particles per μm² per frame. D_frame is linked to the interparticle distance distribution that is implicitly related to the probability p of finding a single particle at a radius r for randomly distributed particles, as determined using the following equation:

$$p\left(r,D_{frame}\right)=\exp \left(-D_{frame}\pi r^2\right).$$ (5)

A scene with two particles recapitulates the problem by identifying the intrinsic limits of the LD/HD and HD/NR transitions (Fig. 1) to design a heuristic adapted to diverse experimental data. We therefore undertook a theoretical study using two particles, P₁ and P₂, of respective intensities α₁ and α₂, and separated from each other by d, the interparticle distance expressed in r₀, the characteristic PSF dimension.

Limit of the LD hypothesis

According to the LD hypothesis, the localization precision is governed by a photon-counting analysis regardless of the local particle density (16). However, the omission of surrounding particles introduces a bias, i_b, into the localization of particle P₁. This value represents a difference between the ground truth position and the localization of a particle. Thus, it is important to determine the distance and contrast at which a surrounding particle P₂ has an effect on the localization precision of P₁.

By considering a signal-dependent noise model (35) (Supporting Materials and Methods, Note S1), i_b may be solved for an MMSE estimator within an analysis square window of dimension w as follows (Supporting Materials and Methods, Note S2):

$$f_{\text{MMSE}}\left(i_b;d,r_0,\frac{{\alpha }_2}{{\alpha }_1}\right)=\sum _{i\in w}\left(i-i_b\right)\exp \left(-\frac{{\left(i-i_b\right)}^2+i^2}{2r_0^2}\right)\left(1+\frac{{\alpha }_2}{{\alpha }_1}\exp \left(-\frac{d\left(d-2i\right)}{2r_0^2}\right)\right)=0.$$ (6)

The LD/HD transition distance at which the bias i_b becomes greater than the expected precision of the localization of ${\text{P}}_1$ also relies on the intensity ratio between the two particles (Fig. 2 a). This result was validated on synthetic data, for which the ${\text{P}}_1$ position was estimated with an MMSE estimator initialized at its true coordinates (Figs. 2 c and S3). We set d_LD/HD, the LD/HD transition distance, at a large dynamic intensity ratio ((α₂/α₁)_dB = 22 dB), a scene we considered conceivable based on experimental data (Fig. 2 b). This transition was found to occur at approximately 5r₀ (i.e., ∼650 nm for a standard r₀ ≈ 130 nm). Thus, any particles spaced at a distance below this limit would require an HD estimation.

Figure 2.

Theoretical limit of the LD hypothesis and its validation on synthetic data. (a) The theoretical bias i_b for P₁ localization using an MMSE estimator under the LD hypothesis at different intensity ratios, α₂/α₁ (w = 9 pixels, r₀ = 1.25 pixels (∼130 nm)), is shown. The sharp transition observed for α₂/α₁ ≥ +1 dB results from the presence of a particle of brighter intensity close to the analyzed one. (b) Synthetic data for two particles at different intensity ratios, α₂/α₁, are shown. (c) The bias i_b for P₁ using an MMSE estimator on synthetic data is shown (see also Fig. S1). α₂/α₁ = −10 dB. Mean ± SD, n = 250 images per d value.

Limit of the HD hypothesis

It was next necessary to determine the distance from which an enumeration is no longer achievable, namely, the HD/NR transition limit (Supporting Materials and Methods, Note S3). This limit, which is only quantifiable on ground truth data, emphasizes that counting particles from such high-density data is flawed. The limit was determined by performing a likelihood ratio test that yields the best probability of detection (PD) calculated for a tolerable level of error set by a PFA (45).

For a simple NR scene (Fig. 1 b), two particles separated by a short distance had an almost null PD at the usual PFA (10⁻⁴). Similarly to LD cases, the use of an erroneous fitting model (i.e., a one-Gaussian fit for a scene with two particles) introduced a bias. This detection error altered the localization of the nearest and more distant particles (Fig. S4 a). Conversely, the detection of any dim particles enabled unbiased localizations (Fig. S4 b). Moreover, the nuisance effect propagated at long distances and depended on both the SNR and the geometry. This observed minimization of the bias in the localization of bright particles implied the need to account for all particles in the detection step, even those with a weak intensity.

Of note, both SNR and D_frame contributed to the assignment of a proper PD. Hence, the HD/NR transition was defined by a detection limit d_HD/NR(SNR) requiring setting to a given SNR, for appropriate PD (80%) and at a weakly restrictive PFA (10⁻³) (Figs. S5 a and S6; Supporting Materials and Methods, Note S3).

A diagram for SMLM accuracy estimation based on density and SNR

Our theoretical study prompted us to link the density and SNR together in a diagram aimed to estimate the localization accuracy achievable on experimental data; the diagram was not intended to cover the experimental complexity of SMLM observations. The transitions distinguish the LD/HD limit at which an HD algorithm is required and the HD/NR limit at which SMLM analyses become incorrect. The isocurves approximate the percentage of particles localized with a given precision.

We chose to plot the isocurves for uniformly distributed particles of equal intensity as, although a more realistic SNR distribution could be considered, the results were found to be comparable (data not shown). Consequently, the diagram defines the precision limits for the simplest case for a large range of realistic D_frame and SNR values.

Density effect on LD, HD and NR subsets

Equation 1 provides the probability of identifying isolated particles for a uniform distribution and inter-particle distances established by the LD/HD and HD/NR transitions. Thus, we defined the partition of the LD, HD, and NR subsets as follows:

$$\{\begin{array}{l}L{D}_{\text{SNR}}\text{\%}=\exp \left(-D_{frame}\pi d_{LD/HD}^2\right)\ast 100\hfill \\ N{R}_{\text{SNR}}\text{\%}=\left(1-\exp \left(-D_{frame}\pi d_{HD/NR}^2\left(\text{SNR}\right)\right)\right)\ast 100\hfill \\ H{D}_{\text{SNR}}\text{\%}=100-LD\text{\%}-NR\text{\%}\hfill \end{array}.$$ (7)

It is important to note that the HD/NR limit established for two particles of equal SNR represents a limit case for any scenes with N particles. Indeed, the achievable precision determined by the CRB for these two particles is lower or equal to that for a scene with N particles.

As specified above, the NR subset depends on SNR. This subset rapidly increased with D_frame up to ∼40% at 5 part/μm²/frame (Fig. 3 a). Although assessable for known ground truth data, it was obviously not for experimental data, leading to significant erroneous quantifications. Thus, we determined the densities/frame corresponding to the respective thresholds d_LD/HD and d_HD/NR(SNR) at a suitable quantile Q_subset set to 20%, referring to the densities at which 20% of the particles have a d value below these thresholds (Fig. S5 a).

Figure 3.

Theoretical D_frame and SNR limits. (a) Color-coded LD, HD, and NR subsets as a function of D_frame with ground truth thumbnails at different densities are shown (SNR = 25 dB, r₀ ≈ 130 nm, and 10⁻⁴ PFA for the HD/NR transition). Note that the NR subset, which increases with D_frame, is only measurable on ground truth data. (b) A density-SNR space diagram is shown. The LD, HD, and NR regions are green, orange, and red color coded, respectively. The LD/HD limit is independent of the SNR. Then, in a green to orange transition, the bias i_b becomes of the same order of magnitude as σ_RMS, the precision of localization. Accurate data analysis then requires an HD algorithm up to a density at which the particles are no longer resolvable (orange to red transition). This limit, which depends on the SNR, is set by the level of PFA stringency acceptable for a given experiment. The isocurves delineate, in the case of uniformly distributed particles, the precision reached by 80% of the particles (Q_σ = 80%). For instance, the thumbnails in (a) are located on the diagram: 87% of particles were localized with a σ_RMS ≤ 17 nm in section (A) and 80% with a σ_RMS ≤ 26 nm in section (B). In section (C), 20% of particles were NR at 10⁻³ PFA. This diagram was generated at known r₀ but estimated α intensity. See Fig. S7 for a comparison with an unknown r₀ or known α hypotheses.

Density and SNR effect on the precision isocurves

We generalized the two particles study to the achievable limits for accurate enumeration and localization in realistic D_frame and SNR ranges and in cases of uniform local distribution of particles. Although the SNR predominantly determines the expected localization precision σ_RMS under LD conditions, this postulate was no longer valid under HD conditions at which increased D_frame decreased σ_RMS (Fig. S5 b). From local density, we identified three limits (Fig. S6; Supporting Materials and Methods, Note S3):

• 1)a “precision limit” when the error became greater than the expected localization precision. We would like to point out that calculating the CRB at known signal intensity or at least with a prior knowledge of its value had no impact on the localization precision for LD data. This was not the case for HD data, because the CRB had been calculated in previous studies without estimating the particle intensity (17, 23, 46), leading to an overestimation of the actual resolvable D_frame. We in fact demonstrated that only a very precise knowledge of the particle intensity would improve the localization precision and such a requirement is unrealistic for experimental data (Supporting Materials and Methods, Note S4). We therefore calculated the CRB for HD conditions by evaluating both the position and intensity of the particles;
• 2)a “separation limit” when the errors of the particle positions became larger than the distance separating them;
• 3)a “detection limit” when, at a given PFA and reasonable PD, the number of particles could no longer be determined.

At a given SNR, these three limits had the same order of magnitude (Fig. S6). Nevertheless, the detection limit was the limiting factor because a loss of accurate particle detection rendered the scene NR. Based on the cumulative σ_RMS distribution (Fig. S5 b), we arbitrarily set a quantile Q_σ at 80% to plot the precision isocurves, meaning that for a given D_frame, 80% of particles had a σ_RMS less or equal to the precision isocurve value. Of note, the CRB calculated with prior knowledge of the intensity drastically overestimated the precision of localization: for instance, at a density of 2.2 part/μm²/frame, the precision was impaired from ∼25 nm (known α) to ∼60 nm (unknown α).

Merged information in a single density-SNR space diagram

The color-coded LD/HD/NR transitions were merged with the precision isocurves in a single diagram (Fig. 3 b). This density-SNR space diagram generated at known r₀ but unknown α defined the experimental local D_frame and SNR required to achieve a given precision. For example, a 15 nm precision would be achievable at 26 dB under LD conditions (D_frame < 0.15 part/μm²/frame). At 1 part/μm²/frame, this precision would be achievable for particles with SNR ≈ 30 dB. At a twofold higher D_frame, only the particles with high SNR (≥40 dB) would be localized with this precision. The SMLM performance also decreased when r₀ needed estimating (e.g., wide field versus total internal reflection fluorescence microscopies; Fig. S7).

UNLOC—a parameter-free algorithm

Based on our theoretical study, we developed UNLOC, a parameter-free algorithm based on a heuristic for reliable particle localization at variable local density with minimal computational cost (Supporting Materials and Methods, Note S6). UNLOC was also designed to achieve the best performance, as delineated by the CRB, for any particles separated by a distance greater than d_HD/NR(SNR) and without any prior knowledge of the particle intensity. Although UNLOC accounts for spatiotemporal variations in the background, SNR and local particle density, it only requires the PSF size (r₀) of the microscope to build a list of particle localizations.

UNLOC principle

The algorithm is divided into three modules (Fig. S9 a): 1) a detection/estimation module performed in an iterative process (Fig. 4) together with optimization of the number, position and intensity of the particles, and also the PSF size, if requested (Fig. S9 b); 2) a reconnection module to minimize an overrepresentation of particles in the reconstructed image using statistical tracking of the particle history over the frames; 3) an optional module for drift correction and classical image rendering methods.

Figure 4.

Principle of UNLOC. A global heuristic of the UNLOC HD mode used for detection and enumeration is given. Regions of interest in the image containing potential particles are defined through a GLRT, and a first list of particles is created (steps 2–3). A first multi-Gaussian fit is performed (step 4), and a new list of particles is created based on the residues (steps 5–6). After the addition of potential particle steps, supernumerary particles are then deleted by an optimization loop of multi-Gaussian fits and a GLRT on residues to find the number of particles and their respective positions, sizes, and intensities (steps 7–9). The residues (step 5) are used only to detect potential additional particles. All multi-Gaussian fits are performed only on the raw data (steps 4 and 8).

Detection/estimation. This module was designed for both static (fixed cells) and dynamic (live cells) observations. It has two modes:

• 1)an LD mode similar to the one in the multiple-target tracing algorithm (47). This is based on a GLRT at unknown background which is an efficient detector dependent only on the PFA (48);
• 2)an HD mode designed for any data with high local density/frame or complex background or both. The analysis primarily requires an estimation of the background mean and variance to maintain the GLRT robustness. This is ensured by filtering and interpolating the background between the regions of the frame without signals. The list of particles is generated more specifically by iterative addition and deletion of particles after the background estimation step (Fig. 4; see also Fig. S9 b for more detail). In all cases, the final particle positions are determined by a global fitting on the background-corrected raw data to delete supernumerary particles from the list. Compared to DAOSTORM (25), the UNLOC heuristic optimizes the particle enumeration by an up-and-down strategy (38) executed conjointly with the optimization of particle position, intensity, and PSF size. Another UNLOC benefit is performing the particle localization on PSF-deconvoluted frames. A resulting more rapid convergence on the final solution shortens the computation time. It should be noted that the UNLOC heuristic is only valid as long as the PSF is correctly determined. In the general framework, this is done by a correct approximation of the Airy pattern of a microscope PSF by a Gaussian profile. Alternatively, and by keeping the same UNLOC heuristic, implementation of another PSF model remains possible but at higher calculation cost.

Reconnection. In SMLM, the fluorescence emission is intrinsically stochastic and can be recorded over consecutive or disconnected frames. Therefore, a statistical reconnection of the particle history over the frames (Supporting Materials and Methods, Note S6, paragraph SN 6.3) was implemented to avoid an overestimation of the total number of particles in static SMLM (fixed biological samples). The reconnections identified in a restricted number of previous frames were tested using the particles present in the current frame. A reverse test validated the reconnection.

Drift correction, data filtering, and rendering. Drift correction is achieved by two optional methods: 1) automated tracking of fluorescent fiduciary markers over the frames (3) or 2) image cross correlation (49), in which subsequent resolved frames are correlated. Standard data filtering is implemented to remove outliers (e.g., any data with low precision in localization, intensity, SNR, ON-state, etc.). Finally, five classical rendering modes (50) are implemented to reconstruct the SMLM image: 1) a binary mode that represents each localization as a white subpixel in the reconstructed image; 2) an integrated binary mode, in which the intensity values assigned to each subpixel correspond to the number of localizations within each pixel; 3) a time mode encoded in a look-up table as a function of the first temporal appearance of the particle; 4) a mode in which each particle is represented by a Gaussian, the variance of which reflects the localization precision; and 5) an integrated Gaussian mode similar to the previous mode but accounting for the local density.

Benchmarking for SMLM algorithms

Classically, the benchmarking metrics pair the ground truth of synthetic bioinspired data and the estimated particle localizations at a tolerance defined by the radius of an area for the closest localization (it is chosen to equal the FWHM of the PSF) (31). Although these metrics are pertinent under real LD conditions, they are unsolvable at increased D_frame (Fig. S10) due to mismatches arising from NR particles. For instance, at a density of 2 part/μm²/frame, 50% of particles are located at a closer distance than this tolerance radius, providing inconsistent pairings.

To overcome this drawback, we designed two charts able to directly visualize the performance of any algorithms (Figs. 5 and S11–S14): one for assessing the limits for the two-particle scene, and the other for mimicking the density-SNR space diagram. The UNLOC analysis of the interparticle distance chart (Fig. 5 a) unambiguously verifies the principal conclusions obtained from our theoretical study: 1) in LD mode, a bias starts occurring for any particles distant by less than around 5r₀, i.e., d_LD/HD; 2) another bias results from ignoring any dimly fluorescent neighboring particles; and 3) in both LD and HD modes, a counting error occurs when the interparticle distance is below d_HD/NR(SNR). UNLOC analysis of the alphanumeric density-SNR chart illustrated the similarity of the localization precision along an isocurve (Fig. 5 b). Overall, UNLOC provided more reliable estimates on the localization precision and achievable resolution (42) than the standard ThunderSTORM HD algorithm (26) on data with a complex background (Figs. S12 and S14). UNLOC also provided an effective particle localization as compared to algorithms based on local density distribution reconstruction, such as super-resolution radial fluctuations (7) (Fig. S14).

Figure 5.

Performance of UNLOC on synthetic data. The UNLOC analyses are performed at known r₀ and estimated α values. (a) The interparticle distance chart mimics a two-particle scene at different SNR ratios (see Fig. S11). Red and green binary image renderings are for LD and HD UNLOC analyses, respectively, and yellow for similar LD/HD particle localization. As expected, the LD/HD transition limit is visible when d ≈ 5r₀, with higher dispersion of the particle localization for the LD mode. The HD/NR transition limit not only depends on the interparticle distance but also on SNR, as demonstrated by the different SNR ratios. The alphanumeric characters, r₀ scales, and marks are solely present for sake of direct visualization of the scene characteristics and illustrate the result of particle localization image rendering; they are generated with particles at high SNR, low density, and under spatial and temporal conditions uncorrelated with the two-particles scene evaluation. (b) The top row shows as follows: on the left, a graphic representation of the density-space diagram, in the middle, a stack of raw data, and on the right, the ground truth of a chart reproducing the density-SNR space diagram (see Fig. S13 for a readable chart). The middle and bottom rows show as follows: ground truth regions (black) numbered on the density-space diagram with their UNLOC binary rendering images (blue) along the 10 and 20 nm precision isocurves, respectively. The green, orange, and red percentages correspond to those for the LD, HD, and NR subsets, respectively. The blurring effect on characters demonstrates the bias as a result of SNR and local D_frame values along an isocurve. Scale bars, 500 nm.

UNLOC performance on experimental data

We chose to illustrate the UNLOC performance on DNA origami nanorulers, which do not require the control of photoinduced switching conditions. The HiRes 80R and HiRes 20R nanorulers are each made of two ATTO 655-binding sites distant by 80 and 20 nm, respectively. The image stacks were analyzed by UNLOC and ThunderSTORM, for which reconnection, drift correction, and data filtering were optimized to be equivalent to those used for UNLOC. Their respective output results evaluated with the GATTAnalysis software illustrated the robustness of UNLOC: the more difficult the sample is, the better the results obtained by UNLOC compared to ThunderSTORM are (see the 20 nm DNA nanorulers) (Fig. 6).

Figure 6.

Comparative UNLOC and ThunderSTORM performances on DNA origami. DNA nanorulers (left column) with two ATTO 655-binding sites distant by 80 and 20 nm, respectively, are shown. Scale bars, 100 nm. The stacks of ∼25,000 frames/sample were analyzed with either UNLOC (green) in HD mode, at estimated r₀ and α value, a reconnection set at one off-frame, a drift correction by correlation, and data filtering to keep particles with σ_xy < 0.05 pixels, or with optimized ThunderSTORM (red) parameters in a multiple-emitter fitting mode and the conditions of reconnection, drift correction, and data filtering equivalent to those ones used for UNLOC. The results on the pass ratio, i.e., the fraction of the proper measurable nanorulers to the total detected structures (%), the site-to-site distance, and the full width at half maximum (FWHM) of individual sites were evaluated with the GATTAnalysis software.

On cell samples, UNLOC efficiently challenged the variability of background, SNR, or density throughout the stack of experimental acquisitions, as illustrated by the image reconstruction of the actin network in Cos-7 cells (Fig. S16).

In terms of analytical speed, UNLOC analyzed in HD mode the density-SNR space diagram chart in ∼11 min (i.e., ∼20 frames/s or ∼3200 particles/s). For the DNA nanorulers data (∼25,000 frames recorded for each DNA origami), the analyses took ∼30 min as compared to ∼10 hr for ThunderSTORM. For the experimental biological data set of ∼57,000 frames of 512 × 512 pixels with nonhomogeneous background and high variability of local density/frame (Fig. S16), UNLOC performed the task in HD mode within ∼200 min (i.e., ∼5 frames/s or ∼2000 particles/s, for a total of ∼20 × 10⁶ detected particles).

UNLOC performance on clustered particles

Our theoretical study has until now been intended for a uniformly distributed population of particles. Importantly, however, experimental data deviate strongly from these ideal conditions. More generally, any biological study encounters possible high local densities, even at a low density per frame, as well as background variations, which fully justifies using an accurate HD mode of analysis. We have illustrated such a case through the analysis of particles localized by UNLOC in LD versus HD mode for nonhomogeneous distributed data (e.g., molecular clustering) (Fig. 7). The results of the clustering analysis by two different methods (39, 40) are recapitulated in Table S2. A proper quantification of clusters depends on the accuracy of the localization precision, which depends on local density, i.e., the density surrounding the particle in a given frame of the image stack. Of interest, UNLOC can also serve as an efficient localization tool for single-particle tracking and the result further analyzed with the multiple-target tracing algorithm (47).

Figure 7.

LD and HD modes of UNLOC analysis on nonhomogeneous distribution. A synthetic stack of multiple frames mimics clustered ground truth positions with, in green, a representative zoomed-in area of one frame (upper right). Lower panels show reconstructed images of the zoomed-in area analyzed in LD (red dots) or HD modes (blue dots) (see also Table S1). Scale bars, 1 μm (insets, 100 nm).

Conclusions

Here, we have investigated the accuracy and robustness of SMLM analysis by establishing a thorough theoretical study for realistic SNR and density values but, more importantly, without prior knowledge of the particle intensity. The impact of both the SNR and local D_frame analyses has previously been assessed in a broad range of studies based on CRB determined at known intensity (21, 22, 23, 24, 25, 26). This assumption is valid as long as LD conditions are effective (i.e., for a density lower than 0.15 part./μm²/frame) (17). However, it does introduce a noticeable overestimation on the precision of localization for data with high local density/frame or complex background or both. Moreover, our theoretical study has here clearly demonstrated that, compared to an MLE, an MMSE estimator is more robust and simpler to compute on high local density/frame data with a complex background.

We have demonstrated that a bias on the precision of localization of one particle starts occurring when another particle is present at relatively long distance (∼600 nm). Consequently, the detection of particles should tend to be exhaustive regardless of their intensity because their omission biases the localization accuracy of the others. We systematically validated these conclusions on synthetic data for which the ground truth was known. Altogether, our study led to the identification of LD/HD/NR subsets able to dictate the working range of any localization-based algorithms. Of importance, NR SMLM scenes arise quickly with increased local density/frame. For instance, two particles of 25 dB distant by ∼170 nm are indistinguishable. Therefore, any NR scene, which is by definition undeterminable on experimental data, impedes the localization precision, image representation, quantification, and even the benchmarking of algorithms performed with classical metrics.

We have thus provided to our knowledge new tools able to help report accurate quantitative SMLM observations:

• 1)a density-SNR space diagram established for a realistic range of D_frame and SNR, enabling standardized evaluation of the localization accuracy;
• 2)UNLOC, a fast, flexible and efficient parameter-free algorithm for high-density SMLM data available as a user-friendly ImageJ plugin. The UNLOC heuristic ensures optimal localization precision and counting of particles for any SMLM data set with high local density/frame or complex background or both;
• 3)two charts for any SMLM algorithms aiming specifically at providing a list of localizations of single molecules. The reconstructed images provide a visual evaluation of HD performance for which classical benchmarking is erroneous because of the presence of NR particles.

In conclusion, analysis of synthetic data has confirmed the reliability of our theoretical study. By taking into account the characteristic of bias on the precision of localization, we designed UNLOC to be able to successfully reconstruct the experimental data obtained with DNA nanorulers or the actin-based meshwork in a fixed sample with reduced computation time.

Author Contributions

N.B. and D.M. supervised the study and conceived the project. S. Mailfert, A.R., L.B., J.T., and N.B. developed the algorithms and performed the simulations. S.B., S. Mailfert, and S. Monneret designed the optical bench. R.F., M.-C.B., and Y.H. performed experimental observations. And all authors contributed to the interpretation of the data. N.B., S. Mailfert, and D.M. wrote the manuscript.

Editor: Julie Biteen.

Supporting Citations

References (51, 52, 53, 54, 55, 56, 57) appear in the Supporting Material.