A state space based approach to localizing single molecules from multi-emitter images

Abstract

Single molecule super-resolution microscopy is a powerful tool that enables imaging at sub-diffraction-limit resolution. In this technique, subsets of stochastically photoactivated fluorophores are imaged over a sequence of frames and accurately localized, and the estimated locations are used to construct a high-resolution image of the cellular structures labeled by the fluorophores. Available localization methods typically first determine the regions of the image that contain emitting fluorophores through a process referred to as detection. Then, the locations of the fluorophores are estimated accurately in an estimation step. We propose a novel localization method which combines the detection and estimation steps. The method models the given image as the frequency response of a multi-order system obtained with a balanced state space realization algorithm based on the singular value decomposition of a Hankel matrix, and determines the locations of intensity peaks in the image as the pole locations of the resulting system. The locations of the most significant peaks correspond to the locations of single molecules in the original image. Although the accuracy of the location estimates is reasonably good, we demonstrate that, by using the estimates as the initial conditions for a maximum likelihood estimator, refined estimates can be obtained that have a standard deviation close to the Cramér-Rao lower bound-based limit of accuracy. We validate our method using both simulated and experimental multi-emitter images.

1. INTRODUCTION

Single molecule super-resolution techniques have been successful at producing images with a higher resolution than allowed by the diffraction limit. In these techniques, a large number of images are taken of a cellular structure that is labeled with fluorescent molecules. In each of the images, only a small number of relatively well-isolated molecules are in an “on” state, and are expected to be detected. The locations of these molecules are then estimated from each of the images. The final high resolution image of the cellular structure is then reconstructed from all the fluorophore locations obtained from the individual images. The performance of the fluorophore localization algorithm plays a key role in the resolution of the final image. Many fluorophore localization methods are available, and they typically comprise the following separate steps: detection and estimation. In the detection step, fluorophores are identified. In the estimation step, the locations of the identified fluorophores are estimated. In recent years, several methods have been developed to solve the estimation problem. Most of these methods are fitting-based, i.e., they estimate the locations of fluorophores by fitting a point spread function (PSF) model to the image data and finding the location coordinates that minimize the difference between the model and data according to a criterion. For example, Huang et al.¹ addressed the multi-emitter fitting problem using a maximum likelihood estimator that simultaneously localizes multiple fluorophores inside a sub-region of the image. These methods are recommended when accurate noise and PSF models are available. This is, however, often not the case as experimental conditions can be difficult to model. Other localization methods are available that use non-fitting algorithms to solve the estimation problem. For example, Henriques et al.² developed a highspeed reconstruction algorithm that uses a modified center of mass (centroid) algorithm to estimate the locations of the identified peaks and the parameters defining the shapes of those peaks in three dimensions. There is a background bias problem associated with centroid-based methods that affects the performance of these methods adversely. To solve this issue, the virtual window center of mass (VWCM) method has been demonstrated to be a good background-corrected centroid estimator.³ Although centroid-based algorithms are relatively fast and computationally less complex compared to fitting-based algorithms, their results are not as accurate as the results of fitting-based algorithms. The compressive-sensing-based method CSSTORM,⁴ structured sparse model and Bayesian information criterion (SSC-BIC),⁵ and fast localization algorithm based on a continuous-space formulation (FALCON)⁶ are other well-known examples of non-fitting algorithms developed based on sparse support recovery methods.⁷ Among them, CSSTORM has been shown to achieve high accuracy even for multi-emitter images with a high density of 10 emitters/μm².⁴ However, this method depends on solving a large-scale convex problem and is computationally complex.⁸ Another important class of non-fitting algorithms estimates the fluorophore locations by transferring the localization problem to the frequency domain. As an example, a localization algorithm has been developed⁸ based on a two-dimensional (2D) spectrum-estimation method called matrix enhancement and matrix pencil (MEMP), which provides a significant speed advantage over CSSTORM while producing the same accuracy. However, this MEMP-based algorithm assumes that a Gaussian model can approximate the PSF, which in practice is not always an accurate assumption.

Here, we propose a novel non-fitting method for single molecule localization which combines the detection and estimation steps. The basis of our proposed method is to model a single molecule fluorescence image, which contains multiple peaks of intensity corresponding to emitting fluorophores, by the frequency response of a multi-order system. For this purpose, we utilize a balanced state space realization algorithm used previously^9–11 for the reduction of noise in fluorescence microscopy images. This algorithm is based on the singular value decomposition (SVD) of a Hankel matrix. The pole locations of the multi-order system then correspond to the peak locations in the frequency domain, or equivalently, the locations of intensity peaks in the original image. The locations of single molecules correspond to the locations of the most significant peaks which are determined through a procedure that utilizes a least-squares criterion.

We assess the performance of the algorithm using both simulated and experimental data. With simulated data, we evaluate the detection rate of the algorithm given molecules with different mean photon counts and separated by different distances. We also simulate repeat images of single molecules, in order to analyze the bias of the algorithm by looking at the average of the deviations of the estimated molecule locations from the ground truth. In the case of one molecule in the given image, we observe that there is no systematic bias associated with the algorithm. However, when multiple molecules are present in the image, our results suggest the existence of bias which depends on the distances between the molecules relative to the image size. Moreover, in the case of repeat images, we analyze the accuracy of the algorithm by looking at the standard deviation of the estimates. Using repeat images of one molecule, we also compare the standard deviation of the estimates with the limit of the localization accuracy, a theoretical accuracy benchmark given by the square root of the Cramér-Rao lower bound (CRLB).¹² Although the accuracy of the algorithm is reasonable, the difference between the accuracy and the limit of accuracy is nevertheless around twice the limit of accuracy in most cases. To improve the accuracy of the estimates, we use the locations estimated using our algorithm as the initial conditions for a maximum likelihood estimator, and we show that the standard deviation of the obtained estimates is much smaller and, in fact, comes close to the limit of accuracy. We further apply the algorithm on an image of Alexa Fluor 647 dye molecules and demonstrate that the algorithm is able to recover the significant intensity peaks which correspond to the single molecules.

This paper is organized as follows. In Sec. 2, viewing a single molecule image as a finite 2D sequence in the frequency domain, we show the existence of minimal and asymptotically stable systems that realize this sequence in the frequency domain. We also explain the overall proposed approach that is developed based on these systems. In Sec. 3, we explain the algorithm in more detail and develop a state space-based localization algorithm based on the SVD of a Hankel matrix. Section 4 is devoted to specifying the parameters used to generate simulated image data, and to describing the procedure of experimental data acquisition. Finally, in Sec. 5, we present the results of applying our proposed algorithm to simulated and experimental single molecule image data, and provide a comprehensive discussion.

Note that our state space localization method has recently been reported elsewhere.¹³ Here, we provide a similar presentation of the study, but apply the algorithm on different simulated and experimental data. In particular, the simulated image data sets considered here have a different background level of 30 photons per pixel, and we find the results to be similar to the results reported previously.¹³

2. SYSTEM IDENTIFICATION USING FREQUENCY MEASUREMENTS

In this section, by viewing single molecule images as finite 2D sequences in the frequency domain, our goal is to show the existence of minimal and asymptotically stable systems that realize such sequences. First, in Lemma 1, we show the existence of minimal and asymptotically stable systems that realize one-dimensional (1D) sequences in the frequency domain. To prove Lemma 1, we take advantage of Proposition 1,¹⁴ which states that finite 1D data sets can be expressed as the impulse response of minimal and asymptotically stable systems, and we additionally make use of a modified version of the subspace-based method developed by McKelvey et al.¹⁵ We then generalize in Theorem 1 the results of Lemma 1 to 2D finite sequences.

Proposition 1

For positive integer N, let X(n) ∈ ℂ^p×m, p, m ∈ ℕ, n = 1, 2, …, N, be a 1D sequence. Then, there exists a minimal and asymptotically stable system (A, B, C), such that

$$X(n)=C{A}^{n-1}B,n=1,2,\dots ,N.$$ (1)

In the following lemma, we show the existence of a minimal and asymptotically stable system that realizes a finite 1D sequence in the frequency domain.

Lemma 1

Let $\stackrel{\sim }{X}\left(k\right)\in ℝ$, k = 1, 2, …, N, be a finite 1D sequence. For n = 1, 2,…, N, let $X\left(n\right):=\left(\mathit{IDFT}\left(\stackrel{\sim }{X}\right)\right)\left(n\right)=\frac{1}{N}{\displaystyle {\sum }_{k=1}^N\stackrel{\sim }{X}\left(k\right)}{e}^{i2\pi kn/N}$ be the inverse discrete Fourier transform (inverse DFT, or IDFT) of $\stackrel{\sim }{X}$. Then, there exists a minimal and asymptotically stable system (A, B, C), such that

$$X(n)=C{A}^{n-1}B,n=1,2,\dots ,N.$$ (2)

Moreover,

$$\stackrel{\sim }{X}(k)=\stackrel{\sim }{C}{\left(e^{i2\pi k/N}I-\stackrel{\sim }{A}\right)}^{-1}\stackrel{\sim }{B},k=1,2,\dots ,N,$$ (3)

where $\stackrel{\sim }{A}$:=A, $\stackrel{\sim }{B}:=(I-A^N)B,\stackrel{\sim }{C}=C$. If A^N = 0, then $\left(\stackrel{\sim }{A},\stackrel{\sim }{B},\stackrel{\sim }{C}\right)$= (A, B, C).

Proof. Let $\stackrel{\sim }{X}\left(k\right)\in ℝ,$ k = 1, 2,…,N, be a finite 1D sequence. Let

$$X(n):=\left(\mathit{IDFT}\left(\stackrel{\sim }{X}\right)\right)(n)=\frac{1}{N}{\displaystyle \sum _{k=1}^N\stackrel{\sim }{X}}(k)e^{i2\pi kn/N},n=1,2,\dots ,N,$$ (4)

be the IDFT of $\stackrel{\sim }{X}$ Then, according to Proposition 1, there exists a minimal and asymptotically stable system (A, B, C), such that

$$X(n)=C{A}^{n-1}B,n=1,2,\dots ,N.$$ (5)

According to Eqs. (4) and (5), we have, for k =1, 2, …, N,

$$\begin{array}{l}\stackrel{\sim }{X}(k)=\left(\mathit{DFT}(X)\right)(k)\\ ={\displaystyle \sum _{n=1}^{N}X(n)e^{-i2\pi kn/N}}\\ =CB{e}^{-i2\pi k/N}+\mathit{CAB}{e}^{-i4\pi k/N}+\dots +C{A}^{N-1}B{e}^{-i2\pi kN/N}\\ =C{e}^{-i2\pi k/N}\left(I+A{e}^{-i2\pi k/N}+\dots +A^{N-1}{e}^{-i2\pi k\left(N-1\right)/N}\right)B\\ =C{e}^{-i2\pi k/N}\left[{\displaystyle \sum _{n=0}^{N-1}{\left(A{e}^{-i2\pi kn/N}\right)}^n}\right]B.\end{array}$$ (6)

For a square matrix T ∈ ℂ^m×m,m ∈ ℕ, where the number 1 is not an eigenvalue of T, we have the identity ${\displaystyle {\sum }_{n=0}^{N-1}{T}^n=}{\left(I-T\right)}^{-1}\left(I-T^N\right)$. Then, since the realization (A, B, C) is asymptotically stable, i.e., |λ(A)|<1 holds for any eigenvalue λ(A) of A, the number 1 is not an eigenvalue of $A{e}^{-i2\pi k/N}$, k = 1, …, N (or equivalently, $I-A{e}^{-2\pi k/N}$, k = 1, …, N, is invertible), and ${{\displaystyle {\sum }_{n=0}^{N-1}(A{e}^{-i2\pi k/N})}}^n={(I-A{e}^{-i2\pi k/N})}^{-1}(I-A^N)$. Substituting this expression into Eq. (6), we have, for k = 1, 2, …, N,

$$\begin{array}{l}\stackrel{\sim }{X}(k)=C{e}^{-i2\pi k/N}{\left(I-A{e}^{-i2\pi k/N}\right)}^{-1}\left(I-A^N\right)B\\ =C{\left(e^{i2\pi k/N}I-A\right)}^{-1}\left(I-A^N\right)B\\ =\stackrel{\sim }{C}{\left(e^{i2\pi k/N}I-\stackrel{\sim }{A}\right)}^{-1}\stackrel{\sim }{B},\end{array}$$ (7)

where $\stackrel{\sim }{A}$:A, $\stackrel{\sim }{B}:=\left(I-A^N\right)B,\stackrel{\sim }{C}=C$. If A^N = 0, then $\left(\stackrel{\sim }{A},\stackrel{\sim }{B},\stackrel{\sim }{C}\right)$= (A, B, C).

In the following theorem, we extend the results obtained for 1D sequences to 2D sequences.

Theorem 1

Let $\stackrel{\sim }{X}\left(k_1,k_2\right)\in ℝ,$ k_i = 1, 2,…,N_i,i= 1,2, be a finite 2D sequence. For n_i = 1, 2,…, N_i, i = 1, 2, let

$$X\left(n_1,n_2\right):=\left(\mathit{IDF}{T}_{2D}(\stackrel{\sim }{X})\right)\left(n_1,n_2\right)=\frac{1}{N_1{N}_2}{\displaystyle \sum _{k_1=1}^{N_1}{\displaystyle \sum _{K_2=1}^{N_2}\stackrel{\sim }{X}}\left(k_1,k_2\right)}{e}^{i2\pi \left(k_1{n}_1/N_1+k_2{n}_2/N_2\right)}$$ (8)

be the inverse 2D DFT of $\stackrel{\sim }{X}$. Then, there exist minimal and asymptotically stable systems (A_i, B_i, C_i), i = 1, 2, such that

$$X\left(n_1,n_2\right)=X_1(n_1)X_2(n_2),n_i=1,2,\dots ,N_i,i=1,2,$$ (9)

where, for i = 1, 2,

$$X_i\left(n_i\right):=C_i{A}_i^{n_i-1}{B}_i,n_i=1,2,\dots ,N_{i.}$$ (10)

Moreover, for k_j = 1, 2, …, N_j, j = 1, 2,

$$\stackrel{\sim }{X}\left(k_1,k_2\right):={{\displaystyle \prod _{j=1}^2{\stackrel{\sim }{C}}_j\left(e^{i2\pi k_j/N_j}I-{\stackrel{\sim }{A}}_j\right)}}^{-1}{\stackrel{\sim }{B}}_j,$$ (11)

where ${\stackrel{\sim }{A}}_j:=A_j,{\stackrel{\sim }{B}}_j:=(I-A_j^{N_j})B_j,{\stackrel{\sim }{C}}_j:=C_j$. For j = 1, 2, if $A_j^{N_j}=0$, then $\left({\stackrel{\sim }{A}}_j,{\stackrel{\sim }{B}}_j,{\stackrel{\sim }{C}}_j\right)=(A_j,B_j,C_j)$.

Proof. Let $\stackrel{\sim }{X}\left(k_1,k_2\right)\in ℝ,k_i=1,2,\dots ,N_i,i=1,2,$, i =1, 2, be a finite 2D sequence. For n_i = 1, …, N_i, i =1, 2, let

$$X\left(n_1,n_2\right):=\left(\mathit{IDF}{T}_{2D}(\stackrel{\sim }{X})\right)\left(n_1,n_2\right)=\frac{1}{N_1{N}_2}{\displaystyle \sum _{k_1=1}^{N_1}{\displaystyle \sum _{k_2=1}^{N_2}\stackrel{\sim }{X}\left(k_1,k_2\right)e^{i2\pi \left(k_1{n}_1/N_1+k_2{n}_2/N_2\right)},}}$$ (12)

be the inverse 2D DFT of $\stackrel{\sim }{X}$. Arrange the entries of X to form a matrix Q as

$$Q:=\left[\begin{array}{cccc}X(1,1)& X(1,2)& \cdots & X(1,N_2)\\ X(2,1)& X(2,2)& \cdots & X(2,N_2)\\ ⋮& ⋮& \ddots & ⋮\\ X(N_1,1)& X(N_1,2)& \cdots & X(N_1,N_2)\end{array}\right]$$ (13)

Decompose Q via SVD as Q = UΣV, where for $r\in \mathbb{N},U\in {ℂ}^{N_1\times r}$ and $V\in {ℂ}^{r\times N_2}$. For n_i= 1, 2, …, N_i, i =1, 2, define $X_1\left(n_1\right)\in {ℂ}^{1\times r}$ and $X_2\left(n_2\right)\in {ℂ}^{1\times r}$, such that

$$\left[\begin{array}{c}{X}_1(1)\\ X_1(2)\\ ⋮\\ X_1(N_1)\end{array}\right]:=U{\sum }^{1/2},\left[X_2(1)X_2(2)\cdots X_2(N_2)\right]:={\sum }^{1/2}V.$$ (14)

Then

$$X(n_1,n_2)=X_1(n_1)X_2(n_2),n_i=1,2,\dots ,N_i,i=1,2.$$ (15)

Moreover, according to Proposition 1, there exist minimal and asymptotically stable systems (A_i, B_i,C_i),i = 1, 2, such that, for i = 1, 2,

$$X_i(n_i)=C_i{A}_i^{n_i-1}{B}_i,n_i=1,2,\dots ,N_i.$$ (16)

According to Eqs. (12) and (15),

$$\begin{array}{l}\stackrel{\sim }{X}(k_1,k_2)=\left(DF{T}_{2D}(X)\right)(k_1,k_2)\\ ={\displaystyle \sum _{n_1=1}^{N_1}{\displaystyle \sum _{n_2=1}^{N_2}X}}(n_1,n_2)e^{-i2\pi \left(k_1{n}_1/N_1+k_2{n}_2/N_2\right)}\\ =\left({\displaystyle \sum _{n_1=1}^{N_1}{X}_1(n_1)e^{-i2\pi k_1{n}_1/N_1}}\right)\left({\displaystyle \sum _{n_2=1}^{N_2}{X}_2(n_2)e^{-i2\pi k_2{n}_2/N_2}}\right)\\ ={\stackrel{\sim }{X}}_1(k_1){\stackrel{\sim }{X}}_2(k_2),K_i=1,2,\dots ,N_i,i=1,2,\end{array}$$ (17)

where ${\stackrel{\sim }{X}}_i\left(k_i\right)=\left(\mathit{DFT}\left(k_i\right)\right)\left(k_i\right),k_{i=1,2,\dots ,}{N}_i,i=1,2$. Then, according to Lemma 1, for k_j = 1, 2,, …, N_j, j = 1, 2,

$${\stackrel{\sim }{X}}_j(k_j):={\stackrel{\sim }{C}}_j{\left(e^{i2\pi kj/N_j}I-{\stackrel{\sim }{A}}_j\right)}^{-1}{\stackrel{\sim }{B}}_j,$$ (18)

where ${\stackrel{\sim }{A}}_j:=A_j,{\stackrel{\sim }{B}}_j:=(I-A_j^{N_j})B_j,{\stackrel{\sim }{C}}_j:=C_j$. For j = 1, 2, if $A_j^{N_j}=0$, then $\left({\stackrel{\sim }{A}}_j,{\stackrel{\sim }{B}}_j,{\stackrel{\sim }{C}}_j\right)=\left(A_j,B_j,C_j\right)$. □

Assume $\stackrel{\sim }{X}\left(k_1,k_2\right)\in ℝ,k_1=1,2,\dots ,N_i,i=1,2$, is the acquired image of single molecules. Once we have system matrices (A_i, B_i,C_i),i = 1, 2, which realize $\stackrel{\sim }{X}$ (Eq. (11)), if we diagonalize A₁ and A₂, then the diagonal elements of the resulting diagonal matrices ${\overline{A}}_1$ and ${\overline{A}}_2$ provide the poles of the system which correspond to the peaks in the image. In the following, we explain this diagonalization process in more mathematical detail.

For $s_1{s}_2\in \mathbb{N}$, if $A_i\in {ℂ}^{s_i\times s_i},i=1,2$ is diagonalized, i.e., if for some invertible $T_i\in {ℂ}^{s_i\times s_i}$, we have the diagonal matrix ${\overline{A}}_i:=T_i{A}_i{T}_i^{-1}=\text{diag}\left(a_1^i,\cdots ,a_{s_i}^i\right)$, $a_{t_i}^i\in ℂ$, t_i = 1, 2, …, s_i,i = 1, 2, then with ${\overline{B}}_i:=T_i{B}_i={[b_1^i,\cdots ,b_{s_i}^i]}^T$, ${\overline{C}}_i:=C_i{T}_i^{-1}=[c_1^i,\cdots ,c_{s_i}^i]$, i =1,2, where $b_{t_1}^1\in {ℂ}^{1\times r}$, $b_{t_2}^2\in ℂ$, $c_{t_1}^1\in ℂ$, $c_{t_2}^2\in {ℂ}^{r\times 1}$, t_i = 1, 2, …, s_i, i = 1, 2, we can, for kj = 1, 2, …, N_j, j = 1, 2, and using Eq. (11), write $\stackrel{\sim }{X}$ in terms of the poles of the system as

$$\begin{array}{l}\stackrel{\sim }{X}\left(k_1,k_2\right)={\displaystyle \prod _{j=1}^2{\overline{C}}_j{\left(e^{i2\pi kj/N_j}I-{\overline{A}}_j\right)}^{-1}{\overline{B}}_j}\\ \begin{array}{l}={\overline{C}}_1\left[\begin{array}{cccc}\frac{b_1^1{c}_1^2}{\left(e^{i2\pi k_1/N_1}-a_1^1\right)\left(e^{i2\pi k_2/N_2}-a_1^2\right)}& \frac{b_1^1{c}_2^2}{\left(e^{i2\pi k_1/N_1}-a_1^1\right)\left(e^{i2\pi k_2/N_2}-a_2^2\right)}& \cdots & \frac{b_1^1{c}_{s_2}^2}{\left(e^{i2\pi k_1/N_1}-a_1^1\right)\left(e^{i2\pi k_2/N_2}-a_{s_2}^2\right)}\\ \frac{b_2^1{c}_1^2}{\left(e^{i2\pi k_1/N_1}-a_2^1\right)\left(e^{i2\pi k_2/N_2}-a_1^2\right)}& \frac{b_2^1{c}_2^2}{\left(e^{i2\pi k_1/N_1}-a_2^1\right)\left(e^{i2\pi k_2/N_2}-a_2^2\right)}& \cdots & \frac{b_2^1{c}_{s_2}^2}{\left(e^{i2\pi k_1/N_1}-a_2^1\right)\left(e^{i2\pi k_2/N_2}-a_{s_2}^2\right)}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{b_{s_1}^1{c}_1^2}{\left(e^{i2\pi k_1/N_1}-a_{s_1}^1\right)\left(e^{i2\pi k_2/N_2}-a_1^2\right)}& \frac{b_{s_1}^1{c}_2^2}{\left(e^{i2\pi k_1/N_1}-a_{s_1}^1\right)\left(e^{i2\pi k_2/N_2}-a_2^2\right)}& \cdots & \frac{b_{s_1}^1{c}_{s_2}^2}{\left(e^{i2\pi k_1/N_1}-a_{s_1}^1\right)\left(e^{i2\pi k_2/N_2}-a_{s_2}^2\right)}\end{array}\right]{\overline{B}}_2\\ ={\displaystyle \sum _{l=1}^{s_1}{\displaystyle \sum _{j=1}^{s_2}\frac{c_l^1{b}_l^1{c}_j^2{b}_j^2}{\left(e^{i2\pi k_1/N_1}-a_l^1\right)\left(e^{i2\pi k_2/N_2}-a_j^2\right)}}}.\end{array}\end{array}$$ (19)

In Eq. (19), $a_{t_1}^1$, $a_{t_2}^2$ t_i = 1, …, s_i,i = 1, 2, denote the eigenvalues of A₁, A₂, respectively, which correspond to peaks in the image $\stackrel{\sim }{X}$. Let the 2D sequence $\stackrel{\sim }{X}$ denote the pixel intensities of our N₁ × N₂ image with pixel width Δx and pixel height Δy, obtained by sampling the image at the center of each pixel. Assume $a_{t_j}^j=\left|a_{t_j}^j\right|e^{i{w}_{tj}^j}$, $0\le w_{t_j}^j\le 2\pi$, t_j =1,…, s_j,j = 1, 2. Then, by linearly mapping a 2π × 2π square region in the frequency domain to the region with area N₁ × N₂ pixels in the image space (between the center of the first pixel and the center of the last pixel) and converting from image space units to object space units, the peak locations in the object space are given by, for t_i = 1, …, s_i, i = 1, 2,

$$x_{t_2}:=\frac{\mathrm{\Delta }x{w}_{t_2}^2{N}_1}{2M\pi }+\frac{\mathrm{\Delta }x}{2M},y_{t_1}:=\frac{\mathrm{\Delta }y{w}_{t_1}^1{N}_2}{2M\pi }+\frac{\mathrm{\Delta }y}{2M},$$ (20)

where M > 0 denotes the lateral magnification of the microscope system.

3. ALGORITHM

So far, we have shown the existence of minimal and asymptotically stable systems that realize a finite 2D sequence (single molecule image) in the frequency domain. Also, we have demonstrated that the poles of the resulting systems correspond to the peak locations in the image. In this section, using the balanced state space realization algorithm introduced by Maciejowski,¹⁴ we propose a step-by-step algorithm to calculate such systems, and to determine the locations of the single molecules using the realization.

Algorithm. Let $\stackrel{\sim }{X}\left(k_1,k_2\right)\in ℝ$, k_i = 1, 2, …, N_i, i = 1, 2, represent the acquired image data.

1. Subtract an estimated background level $\widehat{\beta }$, e.g., the average of the values of the boundary pixels of the image $\stackrel{\sim }{X}$, from the image data $\stackrel{\sim }{X}$, and define the background-subtracted image ${\stackrel{\sim }{X}}_{bs}$ as

   $${\stackrel{\sim }{X}}_{bs}\left(k_1,k_2\right):=\stackrel{\sim }{X}\left(k_1,k_2\right)-\widehat{\beta },k_i=1,2,\dots ,N_i,i=1,2.$$ (21)
2. Let X be the 2D IDFT of ${\stackrel{\sim }{X}}_{bs}$, i.e.,

   $$X\left(n_1,n_2\right):=\left(\mathit{IDF}{T}_{2D}\left({\stackrel{\sim }{X}}_{bs}\right)\right)\left(n_1,n_2\right),n_i=1,2,\dots ,N_i,i=1,2.$$ (22)
3. Arrange the entries of X to form a matrix Q as

   $$Q:=\left[\begin{array}{cccc}X\left(1,1\right)& X\left(1,2\right)& \cdots & X\left(1,N_2\right)\\ X\left(2,1\right)& X\left(2,2\right)& \cdots & X\left(2,N_2\right)\\ ⋮& ⋮& \ddots & ⋮\\ X\left(N_1,1\right)& X\left(N_1,2\right)& \cdots & X\left(N_1,N_2\right)\end{array}\right].$$ (23)
   Decompose Q via SVD as Q = UΣV. Let the positive integer r ≤ K, K = min(N₁N₂), denote the number of retained singular values (see Sec. 3.1). Partition $\sum =\mathit{diag}(\widehat{\sum },\widehat{\widehat{\sum }}),\widehat{\sum }\in {ℂ}^{r\times r},U=\left[\widehat{U},\widehat{\widehat{U}}\right],\widehat{U}\in {ℂ}^{N_1\times r}$ and $V=\left[\begin{array}{c}\widehat{V}\\ \widehat{\widehat{V}}\end{array}\right],\widehat{V}\in {ℂ}^{r\times N_2}$. For n_i = 1, 2, …, N_i, i = 1, 2, define $X_1^r\left(n_1\right)\in {ℂ}^{1\times r}$ and $X_2^r\left(n_2\right)\in {ℂ}^{r\times 1}$, such that

   $$\left[\begin{array}{c}{X}_1^r(1)\\ X_1^r(2)\\ ⋮\\ X_1^r(N_1)\end{array}\right]:=\widehat{U}{\widehat{\sum }}^{1/2},\left[X_2^r(1)X_2^r(2)\cdots X_2^r(N_2)\right]:={\widehat{\sum }}^{1/2}\widehat{V}.$$ (24)
4. Construct the Hankel matrices $H_1\in {ℂ}^{\left(N_1+1\right)\times \left(N_1+1\right)r}$, $H_2\in {ℂ}^{\left(N_2+1\right)\times \left(N_2+1\right)}$ as

   $$H_i:=\left[\begin{array}{cccccc}{X}_i^r\left(1\right)& X_i^r\left(2\right)& \cdots & X_i^r\left(N_i-1\right)& X_i^r\left(N_i\right)& 0\\ X_i^r\left(2\right)& X_i^r\left(3\right)& \cdots & X_i^r\left(N_i\right)& 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ X_i^r\left(N_i\right)& 0& \cdots & 0& 0& 0\\ 0& 0& \cdots & 0& 0& 0\end{array}\right],i=1,2,$$ (25)

   where 0 denotes a block of zeros of the corresponding size. For i = 1, 2, decompose H_i via SVD as H_i = U_iΣ_iV_i. Let the positive integers s_i ≤ N_i, i = 1, 2, denote the numbers of retained singular values in the respective SVDs (see Sec. 3.1). For i = 1, 2, partition ${\sum }_i=\text{diag}({\widehat{\sum }}_i,{\widehat{\widehat{\sum }}}_i),{\widehat{\sum }}_i\in {ℂ}^{s_i\times s_i,}{U}_i=\left[{\widehat{U}}_i{\widehat{\widehat{U}}}_i\right],{\widehat{U}}_1\in {ℂ}^{\left(N_1+1\right)\times s_1},{\widehat{U}}_2\in {ℂ}^{\left(N_2+1\right)r\times s_2}$, and $V_i=\left[\begin{array}{c}{\widehat{V}}_1\\ {\widehat{V}}_i\end{array}\right],{\widehat{V}}_i\in {ℂ}^{s_1\times \left(N_1+1\right)r},{\widehat{V}}_2\in {ℂ}^{s_2\times \left(N_2+1\right)}$, conformally. Let $C_1^{r;s_2}\in {ℂ}^{r\times s_2}$, be the first row of ${\widehat{U}}_1{\widehat{\sum }}_1^{1/2}$, respectively. Also, let $B_1^{r;s_1}\in {ℂ}^{s_1\times r}$ and $B_2^{r;s_2}\in {ℂ}^{s_2\times 1}$ be the first r columns of ${\widehat{\sum }}_1^{1/2}{\widehat{V}}_1$ and the first column of ${\widehat{\sum }}_2^{1/2}{\widehat{V}}_2$, respectively. Assuming ${\widehat{U}}_i=\left[\begin{array}{c}{\overline{U}}_1^i\\ ⋮\\ {\overline{U}}_{N_i}^i\\ {\overline{U}}_{N_i+1}^i\end{array}\right]$, i = 1,2, where ${\overline{U}}_{n_1}^1\in {ℂ}^{1\times s_1},{\overline{U}}_{n_2}^2\in {ℂ}^{r\times s_2}$, n_i = 1,…,N_i + 1 define ${\widehat{U}}_i^{\uparrow }:=\left[\begin{array}{c}{\overline{U}}_2^i\\ \\ {\overline{U}}_{N_i+1}^i\end{array}\right]$, ${\widehat{U}}_i^{\downarrow }:=\left[\begin{array}{c}{\overline{U}}_1^i\\ ⋮\\ {\overline{U}}_{N_i}^i\end{array}\right]$, i = 1, 2. Then, let $A_i^{r;s_i}={\widehat{\sum }}_i^{-1/2}{\widehat{U}}_i^{\downarrow \ast }{\widehat{U}}_i^{\uparrow }{\widehat{\sum }}_i^{1/2}\in {ℂ}^{s_i\times s_i}$, i = 1, 2.

5. Diagonalize $A_j^{r;s_j}\in {ℂ}^{s_j\times s_j}$, j = 1, 2, i.e., for $t_j=1,2,\dots ,s_j,j=1,2$, and some invertible $T_j\in {ℂ}^{s_j\times s_j}$, let ${\overline{A}}_j^{r;s_j}:=T_j{A}_j^{r;s_j}{T}_j^{-1}=\text{diag}(a_1^j,\cdots ,a_{s_j}^j),a_{t_j}^j=\left|a_{t_j}^j\right|e^{i{w}_{t_j}^j}\in ℂ,0\le w_{t_j}^j\le 2\pi$, be a corresponding diagonal matrix for $A_j^{r;s_j}$. Also, let ${\overline{B}}_j^{r;s_j}:=T_j{B}_j^{r;s_j}={\left[b_1^j,\cdots ,b_{s_j}^j\right]}^T,$ ${\overline{C}}_j^{r;s_j}:={\overline{C}}_j^{r;s_j}{T}_j^{-1}=\left[c_1^j,\cdots ,c_{s_j}^j\right]$ j = 1, 2, where $b_{t_1}^1\in {ℂ}^{1\times r},b_{t_2}^2\in ℂ,c_{t_1}^1\in ℂ,c_{t_2}^2\in {ℂ}^{r\times 1}$,t_j = 1, 2,…, s_j,j = 1, 2.

6. For h = min(s₁, s₂), calculate, in the object space, the estimated peak locations $\left(x_k,y_k\right),x_k\in \left\{{\widehat{x}}_1,\dots ,{\widehat{x}}_2\right\},y_k\in \left\{{\widehat{y}}_1,\dots ,{\widehat{y}}_{s_1}\right\}$, k = 1,…, h, where

   $${\widehat{x}}_{t_2}:=\frac{\mathrm{\Delta }x{w}_{t_2}^2{N}_1}{2M\pi }+\frac{\mathrm{\Delta }x}{2M},{\widehat{y}}_{t_1}:=\frac{\mathrm{\Delta }y{w}_{t_1}^1{N}_2}{2M\pi }+\frac{\mathrm{\Delta }y}{2M},t_i=1,2,\dots ,s_i,i=1,2,$$ (26)

   where Δx and Δy are the width and height of each pixel of the image, respectively, and M > 0 denotes the lateral magnification of the microscope system.

The important questions that need to be addressed in the algorithm are: “how many singular values should be retained in each SVD?”, and “which estimated peak locations are associated with the single molecule locations in the image?”. In the following subsections, we answer these questions.

3.1 Determination of the number of retained singular values in each SVD, and the number of single molecules in the image

Let σ₁ ≥ … ≥ σ_K ≥ 0, K = min(N₁, N₂), denote the singular values in the first SVD (step III). For r = 1, …, K, let $E_r:={\displaystyle {\sum }_{i=1}^r{\sigma }_i^2}$ be the energy of the sequence ${\sigma }_i$, i =1,…, r. Since most of the singular values resulting from an SVD are relatively small and are considered to correspond to noise,¹¹ here, the idea is to retain only the singular values with high energy levels. Estimate the optimal number of retained singular values r in the first SVD as

$$\widehat{r}=\underset{r=1,\dots ,K}{\min }\left\{r:\frac{E_r}{E_K}>\tau \right\},$$ (27)

where $\tau \in ℝ$ denotes a threshold value typically chosen in the range [0.8, 0.9].⁸

Let ${\sigma }_1^i\ge \dots \ge {\sigma }_{N_i}^i\ge 0$, i = 1, 2, be the singular values in the second and third SVDs (step IV), respectively. For l_i = 1,…, N_i, i = 1,2, let

$${\widehat{l}}_i=\underset{l_i=1,\dots ,N_i}{\min }\left\{l_i:\frac{E_{l_i}}{E_{N_i}}>{\tau }_i\right\},$$ (28)

where $E_{l_i}:={\displaystyle {\sum }_{j=1}^{l_i}{\left({\sigma }_j^i\right)}^2}$ and $E_{N_i}:={\displaystyle {\sum }_{j=1}^{N_i}{\left({\sigma }_j^i\right)}^2}$ are the energies of the sequences ${\sigma }_1^i,\dots ,{\sigma }_{l_i}^i$ and ${\sigma }_1^i,\dots ,{\sigma }_{N_i}^i$, respectively, and ${\tau }_i\in ℝ$ denotes a threshold value which is again typically chosen in the range [0.8, 0.9]. The estimates ${\widehat{l}}_i,i=1,2$, thus denote the number of singular values that remain after discarding those that are considered to obviously correspond to noise. We next try to reduce further the number of singular values to retain using an optimization approach that minimizes the difference between the original image and the image reconstructed from the estimated locations of intensity peaks in the original image.

For $s_i=1,\dots ,{\widehat{l}}_i,i=1$, 2, let ${\stackrel{\sim }{X}}^{r;s_1,s_2}\left(k_1,k_2\right)={\displaystyle {\sum }_{l=1}^{s_1}{\displaystyle {\sum }_{j=1}^{s_2}\frac{c_l^1{b}_l^1{c}_j^2{b}_j^2}{\left(e^{i2\pi k_1/N_1}-a_l^1\right)\left(e^{i2\pi k_2/N_2}-a_l^2\right)}}}$, k_t =1,…,N_t, t = 1, 2, be the image reconstructed via the algorithm (Eq. (19)) by retaining r singular values in the first SVD and s₁ and s₂ singular values in the second and third SVDs, respectively. For the pole $\left({\overline{a}}_k^1,{\overline{a}}_k^2\right),{\overline{a}}_k^t\in \left\{a_1^t,\cdots ,a_{s_t}^t\right\},{\overline{a}}_k^t:=\left|{\overline{a}}_k^t\right|e^{i{\overline{w}}_k^t},0\le {\overline{w}}_k^t\le 2\pi$, k = 1, …, s₁s₂, t= 1,2, and its corresponding product of coefficients in the numerator $p_k\in ℂ,k=1,\dots$, …,s₁s₂, we refer to $\left|\frac{p_k}{\left(e^{i{\overline{w}}_k^1}-{\overline{a}}_k^1\right)\left(e^{i{\overline{w}}_k^2}-{\overline{a}}_k^2\right)}\right|$ as the magnitude of the corresponding peak. In the following, we determine the optimal number of retained singular values in the second and third SVDs, and the peak locations which correspond to the single molecule locations in the image.

For h = min(s₁,s₂), let ${\widehat{\theta }}^h:=\left({\widehat{\theta }}_1,\dots ,{\widehat{\theta }}_2\right)\in {ℝ}^{2h}$, n = 1, …, h, such that

$${\widehat{x}}_n:=\frac{\mathrm{\Delta }x}{2M}\frac{{\overline{w}}_n^2{N}_1}{\pi }+\frac{\mathrm{\Delta }x}{2M},{\widehat{y}}_n:=\frac{\mathrm{\Delta }y}{2M}\frac{{\overline{w}}_n^1{N}_2}{\pi }+\frac{\mathrm{\Delta }y}{2M}$$ (29)

are the estimated locations of the h peaks with the largest magnitudes calculated via the algorithm. In general, we consider all possible h-combinations of the poles of ${\stackrel{\sim }{X}}^{r;s_1,s_2}$, but the single molecules typically correspond to the peaks with the largest magnitudes. Let $\left\{z_1,\dots ,z_{N_{\mathit{ptx}}}\right\}$ denote our acquired data, where N_pix denotes the number of pixels in the image. Then, the optimal numbers ${\widehat{s}}_1$ and ${\widehat{s}}_2$ of retained singular values in the second and third SVDs, respectively, are given by

$$\left({\widehat{s}}_1,{\widehat{s}}_2\right)=\underset{\left(s_1,s_2\right),s_i=1,\dots ,{\widehat{l}}_i,i=1,2}{\arg \min }\left({\displaystyle \sum _{k=1}^{N_{\mathit{pix}}}{\left(z_k-{\mu }_{{\widehat{\theta }}^h}\left(k\right)\right)}^2}\right),$$ (30)

and the estimated number of single molecules $\widehat{h}$ is given by $\widehat{h}$ = min $\left({\widehat{s}}_1,{\widehat{s}}_2\right)$. In Eq. (30), ${\mu }_{{\widehat{\theta }}^h}\left(k\right)$ k = 1,…, N_pix, denotes the mean number of photons detected in the k^th pixel from h assumed molecules. In the case that the single molecule image is modeled with a 2D PSF, ${\mu }_{{\widehat{\theta }}^h}\left(k\right)$, k = 1,…, N_pix, is given by¹²

$${\mu }_{{\widehat{\theta }}^h}\left(k\right)={\displaystyle \sum _{n=1}^h\frac{N_{p,n}}{M^2}}{\displaystyle {\int }_{C_k}q\left(\frac{x}{M}-{\widehat{x}}_n,\frac{y}{M}-{\widehat{y}}_n\right)}\mathit{dxdy},{\widehat{\theta }}^h\in {ℝ}^{2h},h=\min \left(s_1,s_2\right)$$ (31)

where N_p,n is the expected number of photons due to the n^th molecule that impact the detector plane during the image exposure, $C_k\subset {ℝ}^2$ denotes the region in the detector plane occupied by the k^th pixel, and q is the 2D PSF of the optical system. If the PSF is the Airy profile, then q is given by

$$q\left(x,y\right):=\frac{J_1^2\left(\frac{2\pi n_a}{\lambda }\sqrt{x^2+y^2}\right)}{\pi \left(x^2+y^2\right)},\left(x,y\right)\in {ℝ}^2,$$ (32)

where n_a denotes the numerical aperture of the objective lens, λ denotes the emission wavelength of the molecule, and J₁ denotes the first order Bessel function of the first kind.

4. METHODS

4.1 Simulation parameters

To analyze the performance of the proposed algorithm, we simulated different data sets using parameters commonly used in single molecule experiments. Some data sets comprise repeat images of one molecule, and some comprise repeat images of more than one molecule. Also, some data sets are such that each image contains a different set of molecules whose locations are randomly chosen based on uniform distributions that place the molecules within different spatial intervals inside the image. Regardless of the data set, the image of a molecule was generated with the Airy profile of Eq. (32) with a numerical aperture of n_a = 1.4 and an emission wavelength of λ = 485 nm. Furthermore, a lateral magnification of M = 100, a detector pixel size of 6.5 μm × 6.5 μm, and a zero-mean Gaussian readout noise with standard deviation σ = 6 e⁻ per pixel, were assumed. Also, we assumed a background level of β = 30 photons/pixel.

4.2 Imaging experiments

4.2.1 Sample preparation

High-performance Zeiss coverslips (#1.5) were prepared using the following procedure: coverslips were sonicated with 50% HPLC-grade ethanol, 1mM HCl with 50% HPLC-grade ethanol, 1M KOH with 50% HPLC-grade ethanol, and 50% HPLC-grade ethanol in succession, each for 20 minutes. The cleaned coverslips were then attached to MatTek dishes, after which 200 μl of Poly-L-lysine (PLL) solution (Sigma-Aldrich) were added to the glass bottom area of the dishes at room temperature. After 10 minutes, the PPL solution was removed and 250-pM Alexa Fluor 647 fluorescent dye (Invitrogen) in 200 μl of phosphate-buffered saline (PBS) was added, also at room temperature. After 10 minutes, the sample was washed twice with PBS at room temperature, following which 1 ml of PBS was added.

4.2.2 Microscopy setup

Custom laser optics, configured with 635-nm and 405-nm diode lasers (OptoEngine) for the excitation and photoactivation, respectively, of Alexa Fluor 647, were used with a Zeiss Axio Observer.A1 microscope. The lasers were reflected onto the sample using a dichroic filter (Di01-R405/488/561/635-25× 36; Semrock) and focused on the back focal plane of a 63×, 1.46 NA Zeiss objective lens. The fluorescence emitted by Alexa Fluor 647 was collected by the objective lens and filtered with a single bandpass filter (FF01-676/29-25; Semrock). Images were acquired using an electron-multiplying charge-coupled device camera (iXon DU897-BV; Andor) in conventional readout mode. The pixel size of the camera was 16 μm × 16 μm. Custom software written in the C programming language was used to control and synchronize the various components, including the lasers, shutters, and the camera.

4.2.3 Super-resolution imaging

PBS was removed from the Alexa Fluor 647 sample prepared as described in Sec. 4.2.1, and imaging buffer consisting of 50-mM beta-mercaptoethylamine (MEA), 0.5-mg/ml glucose oxidase, and 40-μg/ml catalase in PBS (pH 7.4 with 10% glucose), was added. The sample was then sealed with a coverslip and positioned on the microscope sample stage for 5 to 10 minutes to allow the temperature to equilibrate and the oxygen scavenging process to occur. Images were then acquired at a rate of 20 frames per second. The sample was alternately illuminated with the 635-nm and 405-nm lasers, with photoactivation by the 405-nm laser occurring every third frame. Frames with the illumination by the 405-nm laser were not used in the data analysis.

5. RESULTS AND DISCUSSION

We applied the proposed algorithm to different simulated and experimental data sets. In this section, we show and discuss the results obtained.

5.1 Results for simulated data

In this subsection, for different simulated data sets, we first examine the detection rate of the algorithm. For this purpose, we simulated data sets containing images in which the locations of the molecules were chosen randomly, and applied the algorithm. Then, by pairing the estimated locations with the ground truth values, we calculated detection rate measures to evaluate the results. We also evaluate the bias and accuracy of the algorithm using data sets consisting of repeat images of molecules. The bias of the algorithm is evaluated by comparing the average of the estimates with the ground truth. The accuracy of the algorithm is assessed by looking at the standard deviation of the estimates. In the case where there is only one molecule, we also compare the standard deviation of the estimates with the limit of the localization accuracy given by the square root of the CRLB.

5.1.1 One molecule

To evaluate the performance of the algorithm in terms of the detection rate, we simulated data sets in which each image contains one molecule whose location was randomly determined based on a uniform distribution that places it within the image. For a given data set, the mean photon count is the same for the molecule in every image. Different data sets differ by this mean photon count, which ranges from 500 to 4500. For each mean photon count, we simulated 500 images. To calculate statistical measures of the detection rate, we needed to pair the molecules localized by the algorithm with the molecules from the ground truth. For this purpose, we used the Hungarian algorithm with a search area of radius 100 nm.¹⁶ We categorized the localized molecules which were successfully paired with ground truth molecules as true positives. Ground truth molecules that were not paired with a localized molecule and localized molecules which were not paired with a ground truth molecule were categorized as false negatives and false positives, respectively. Denoting the number of true positives by TP, the number of false negatives by FN, and the number of false positives by FP, we define the precision (PRE) and recall (REC) measures as¹⁶

$$\mathit{PRE}:=\frac{TP}{FP+TP},\mathit{REC}:=\frac{TP}{FN+TP}.$$ (33)

Figure 1 shows, except in the case of the relatively low mean photon count of 500 photons/molecule, the recall is 1. This demonstrates that the algorithm detects no false negatives when relatively large numbers of photons are detected from the molecules. Even in the case of 500 photons/molecule, the recall is still relatively high (around 0.995). Also, the precision is always above 90%, even when the mean number of photons is as low as 500. This demonstrates that a large percentage of detected molecules are true positives.

Figure 1.

Analysis of the detection rate of the algorithm, applied to data sets in which each image contains one molecule whose location in the image is chosen randomly according to a uniform distribution that places it within the image. For a given data set, the same mean photon count is used to simulate the molecule in each image. Different data sets differ by this mean photon count. For each mean photon count, 500 images of size 15 × 15 pixels were simulated using the parameters given in Sec. 4.1. The Hungarian algorithm with a search area of radius 100 nm is used to pair the localized molecules with the ground truth molecules.

To examine the performance of the algorithm in terms of bias, we simulated data sets containing repeat images of one molecule. The data sets differ by the mean photon count of the molecule, which we assume to be the same for all frames in a given data set. This mean photon count ranges from 500 to 4500 for the different data sets. For each data set, we simulated 1000 repeat images. Figure 2 shows, as a function of the mean photon count, the differences between the averages of the x- and y-estimates for the correctly detected (i.e., true positive) molecules and the corresponding true x- and y-coordinates. The estimated bias spreads almost evenly about 0 nm for both coordinates, suggesting that in the case of only one molecule per image, there is no systematic bias associated with our proposed algorithm.

Figure 2.

Analysis of the average of location estimates obtained from repeat images of one molecule. Shown in the left and right plots are the difference between the average of the x-estimates and the true x-value, and the difference between the average of the y-estimates and the true y-value, respectively, for data sets that differ by the mean photon count assumed for the molecule per image. For each mean photon count, the data set consists of 1000 images of size 15 × 15 pixels, simulated using the parameters given in Sec. 4.1.

To evaluate the accuracy of the algorithm, for nine of the data sets from Fig. 2, we calculated the standard deviations of the x-estimates and y-estimates for the correctly detected (i.e., true positive) molecules. The results are shown in the first row of Fig. 3. Also, in the second row of Fig. 3, we show the percentage differences between the standard deviations and the CRLB-based limits of the x-localization accuracy and y-localization accuracy. The percentage difference is the absolute difference between the standard deviation of the estimates and the corresponding limit of accuracy, expressed as a percentage of the limit of accuracy. As shown in Fig. 3, when the mean number of photons increases, the accuracy of the algorithm improves, i.e., the standard deviation of the estimates decreases. Also, as can be seen in Fig. 3, for most mean photon counts, the differences between the standard deviations of the estimates and their respective limits of the localization accuracy are around twice (i.e., around 200% of) the limits of accuracy. In the case of the lowest mean photon count of 500, the percentage difference is even more significant. The relatively large percentage differences are due to the fact that our proposed algorithm approximates the Airy profile here with the frequency response of a first-order system, and the shape of the peak of the Airy profile is not exactly the same as the peak of the first-order system in the frequency domain. In order to improve the accuracy of our estimates, we used the location estimates obtained with the algorithm as initial conditions for the maximum likelihood estimation of the location of the molecule from the same images. This maximum likelihood estimator fits an Airy photon distribution profile to the image data,¹⁷ and we applied it to three of the data sets from Fig. 3. The standard deviations of the resulting x-estimates and y-estimates, and the percentage differences between them and the limits of the x-localization accuracy and y-localization accuracy, are shown in Table 1. As can be seen in the table, the standard deviations are significantly smaller than those obtained with the algorithm (Fig. 3) and, consistent with the results reported previously for maximum likelihood estimation,^{17, 18} they approach their respective limits of accuracy.

Figure 3.

Analysis of the standard deviation of location estimates obtained from repeat images of one molecule. Shown in the first row are the standard deviations of the x- and y-estimates for nine of the data sets from Fig. 2. In the second row, we show the percentage difference between the standard deviation of the x-estimates and the limit of the x-localization accuracy, and the percentage difference between the standard deviation of the y-estimates and the limit of the y-localization accuracy. The percentage difference is the absolute difference between the standard deviation of the estimates and the corresponding limit of accuracy, expressed as a percentage of the limit of accuracy.

Table 1.

Analysis of the standard deviation of location estimates produced by the maximum likelihood estimator when the location estimates obtained with the algorithm are used as the initial conditions. Results are shown for the data sets from Fig. 3 with mean photon counts of 500, 2500 and 4500.

Data set	Mean photon count	Standard deviation (SD) of x-estimates (nm)	% difference between SD of x-estimates and x-localization accuracy	Standard deviation (SD) of y-estimates (nm)	% difference between SD of y-estimates and y-localization accuracy
1	500	8.599	2.59	8.893	0.73
2	2500	2.434	2.74	2.266	4.34
3	4500	1.532	1.41	1.659	6.75

5.1.2 Multiple molecules

To analyze the detection rate of the algorithm for data sets with multiple molecules in each frame, we simulated data sets containing images of two closely spaced molecules. For each image, the location of each molecule is randomly chosen from a uniform probability distribution that places the molecule inside the image, such that the distance between the two molecules is not less than a minimum distance d_min. In one case, all data sets are simulated with the same minimum distance of d_min = 100 nm, but differ by the mean photon count per molecule, which ranges from 500 to 4500. In another case, the mean photon count is the same for all data sets at 2500 photons/molecule, but the data sets differ by d_min, which ranges from 100 nm to 500 nm. Figure 4 shows the precision and recall measures for the different data sets (we again use the Hungarian algorithm with a search area of radius 100 nm to pair the localized molecules with the ground truth molecules). As can be seen, except in the case of the relatively low mean photon count of 500 photons/molecule, the recall is around 1. This demonstrates that the number of false negatives are negligible when relatively large numbers of photons are detected from the molecules. Note that even in the case of 500 photons/molecule, the recall is still reasonable (around 0.9). The figure shows that the precision is likewise quite good and for all data sets except one, it is greater than 0.95, i.e., more than 95% of the detected molecules are true positives. For the data set with the lowest mean photon count of 500 per molecule, the precision is smaller (around 0.8). Note that we also analyzed the detection rate for data sets with 3 and 5 molecules per image, and obtained similar results.

Figure 4.

Analysis of the detection rate of the algorithm when applied to data sets in which each image contains two molecules whose locations in the image are chosen randomly. For a given data set, the mean photon count is the same for each molecule in every frame. The location of each molecule is drawn from a uniform distribution that places it inside the image, with the constraint that the distance between each pair of molecules is not less than the minimum distance d_min. For each data set, we simulated 200 images of size 30 × 30 pixels using the parameters given in Sec. 4.1. The precision and recall measures are shown as a function of d_min in the left plot, where the mean photon count is 2500 photons/molecule, and are shown as a function of the mean photon count in the right plot, where d_min = 100 nm. The Hungarian algorithm with a search area of radius 100 nm is used to pair the localized molecules with the ground truth molecules.

We next analyze the bias and accuracy of the algorithm in the case of images with multiple molecules. Unlike the one-molecule case, we observe a bias which depends on the distance between the molecules in relation to the image size. To fully characterize the bias, we simulated data sets comprising 15 × 15-pixel, 20 × 20-pixel, and 40 × 40-pixel images of two molecules in order to cover different combinations of the distance d between the molecules compared to the image size. For a given data set, we simulated 500 images with a mean photon count of 2500 photons/molecule, a pixel size of 6.5 μm × 6.5 μm, and a lateral magnification of M = 100. Then, the area occupied by an N × N-pixel image in the object space is an s × s square region, where s = 65N nm. For each data set, the difference between the average of the estimated x-locations of the correctly detected molecules and the corresponding true x-coordinate is plotted in Fig. 5 for both molecules. The figure shows that, for each data set considered, the minimum of the estimated bias occurs when the distance d between the two molecules is equal to s/2 (e.g., for N=20, s/2 = 650 nm). When d = s/2, the difference between the phases of the poles of the second-order system resulting from the algorithm approaches the maximum of π rad on the unit circle, thus minimizing the mutual effect of the poles on each other. Similar results were obtained from an analysis of the y-estimates. Note that we also repeated the same analysis for data sets with 3 and 5 molecules per image, and obtained similar results.

Figure 5.

Analysis of the average of the location estimates obtained from sets of repeat images of two molecules. Shown in the left and right plots are the differences between the average of the x-estimates and the true x-value for the first and second molecules, respectively, for data sets comprising 15 × 15-pixel, 20 × 20-pixel, and 40 × 40-pixel images. For each image size, distances d between the two molecules are chosen such that distances around s/2, where s = 65N nm is the side length of the square region occupied by an N × N-pixel image in the object space, are represented. For a given data set, we simulated 500 images with a mean photon count of 2500 photons/molecule and the parameters given in Sec. 4.1. The results for d = s/2 are shown with filled symbols.

As we did in the case of one molecule, we next analyze the bias and accuracy of the algorithm as a function of the mean photon count per molecule. For this purpose, we simulated data sets that contain repeat images of two molecules such that the distance d between them is 650 nm. These data sets again differ by the mean photon count per molecule, which we assume does not vary from frame to frame. The mean photon count per molecule ranges from 500 to 4500 for the different data sets. Each data set contains 500 20 × 20-pixel images. To assess the bias of the algorithm, for each molecule in a given data set we calculated the difference between the average of the estimated x-locations and the corresponding true x-coordinate. As can be seen in Fig. 6, the estimated bias is evenly spread around 0 nm, which is consistent with the illustration of bias in Fig. 5 when d = 650 nm for 20 × 20-pixel images. Also, we calculated the standard deviations of the estimated x-locations for nine of the data sets. As shown in Fig. 6, as the mean number of photons per molecule increases, the standard deviation of the estimates decreases. Note that we also analyzed the y-estimates and obtained similar results.

Figure 6.

Analysis of the average and standard deviation of location estimates obtained from sets of repeat images of two molecules as a function of the mean photon count per molecule. The distance d between the two molecules is 650 nm. The data sets differ by the mean photon count per molecule. For each mean photon count, the data set consists of 500 repeat images of size 20 × 20 pixels, simulated using the parameters given in Sec. 4.1. Shown in the first row are the differences between the average of the estimated x-locations and the corresponding true x-coordinates for the two molecules. In the second row, we show the standard deviations of the estimated x-locations for the two molecules.

5.2 Results for experimental data

Here, we apply the proposed algorithm to an 80 × 80-pixel experimental super-resolution image of Alexa Fluor 647 dye molecules acquired as described in Sec. 4.2. In Fig. 7, we show the magnitude of the reconstructed image obtained from the algorithm. As can be seen, using the algorithm, we were able to recover the locations of the significant intensity peaks in the original image that are associated with the locations of the dye molecules. Also, in order to make a better visual comparison between the peaks of the reconstructed image and the original image, we applied the algorithm to a relatively small 30 × 30-pixel region of interest (ROI) of the original dye molecule image. Both the acquired and the reconstructed versions of this 30 × 30-pixel ROI are shown in Fig. 8.

Figure 7.

Result of the algorithm when applied to an experimental super-resolution image. (a) Image of individual Alexa Fluor 647 molecules acquired using the microscopy setup described in Sec. 4.2. The pixel size and image size are 16 μm × 16 μm and 80 × 80 pixels, respectively. (b) The magnitude of the reconstructed image obtained with the algorithm.

Figure 8.

Result of the algorithm when applied to an ROI from an experimental super-resolution image. (a) A 30 × 30-pixel ROI of the super-resolution image shown in Fig. 7. (b) The magnitude of the reconstructed image (algorithm result). (c) and (d) show the mesh plots of the images in (a) and (b), respectively.