Information optimization of laser scanning microscopes for real-time feedback-driven single particle tracking

Abstract

Real-time feedback-driven single particle tracking (RT-FD-SPT) is a class of microscopy techniques that uses measurements of finite excitation/detection volume in a feedback control loop to actuate that volume and track with high spatio-temporal resolution a single particle moving in three dimensions. A variety of methods have been developed, each defined by a set of user-defined choices. Selection of those values is typically done through ad hoc, off-line tuning for the best perceived performance. Here we present a mathematical framework, based on optimization of the Fisher information, to select those parameters such that the best information is acquired for estimating parameters of interest, such as the location of the particle, specifics of the excitation beam such as its dimensions or peak intensity, or the background noise. For concreteness, we focus on tracking of a fluorescently-labeled particle and apply this framework to determine the optimal parameters for three existing fluorescence-based RT-FD-SPT techniques with respect to particle localization.

1. Introduction

Single particle tracking (SPT) has become a common tool in the study of biophysical processes at the molecular level [1–4]. The ability to label and infer the location of single particles and molecules with nanometer resolution has unlocked the ability to study a wide variety of biophysical phenomena at the single molecule scale, including the motion and function of molecular motors [5], the mechanisms of viral entry [6], and neuropathological disease [7]. As with the development of any tool set, its success also reveals new questions that lie outside existing capabilities, driving the need for further improvement in performance. In particular, there is a desire to observe both subtle, fast, and transient phenomenon at the characteristic time scales of molecular processes [8] within the three dimensional environment of the cell. This necessitates an increase in spatio-temporal resolution, longer tracking durations, and tracking in three dimensions over large ranges.

A new class of SPT microscope has emerged to address these challenges. Real-time feedback-driven single particle tracking (RT-FD-SPT) microscopes are systems that track individual labeled particles and molecules through the combination of non-imaging based scanning and feedback control [9,10]. Spatio-temporally varying excitation patterns, such as orbital scanning, provide localization resolution down to the 10 nanometer range [11] and temporal resolution down to 20 microseconds. Additionally, these microscopes are not limited by the depth of field and field of view, providing tracking in three dimensions at up to a few centimeters in range [12]. These advances have enabled single molecule measurements beyond just tracking, such as locking-on to a transcription site on the genome [13], anti-bunching measurements of freely diffusing quantum dots [14], and measuring viral first contacts [15].

As illustrated in Fig. 1, RT-FD-SPT methods are built around the core idea of using the photon measurements in real time in a feedback controller. In general, tracking starts by scanning the laser and co-aligned detection through a local scanning trajectory such as a measurement constellation [11,16], an orbital scan [14,17–19] (as illustrated in Fig. 1(b), top), a local raster scan [19,20], or a knight’s tour [21]. During this scan, a single photon counting detector measures fluorescence photons (Fig. 1(c), top) and an online estimation algorithm translates these photon detections into an estimate of the particle’s position (Fig. 1(b), bottom). The estimated position is fed back into a controller which adjusts the location of the next scan so as to drive the difference between the center of the scan and the (estimated) position of the particle to zero. This effectively locks the laser onto the moving particle (Fig. 1(c), bottom), enabling long duration, high spatio-temporal resolution observations of molecular processes. There is also a sub-class of algorithms that avoid the need for online estimation by using the photon counts directly to center the particle in the scan [22]; the essential idea, however, remains the same.

Fig. 1.

Illustration of an RT-FD-SPT laser scanning microscope and its operation. (a) Depiction of a single detector RT-FD-SPT microscope. The arrows indicate the signal path for the control system. Inset is a zoom showing the excitation laser (blue) exciting output fluorescence (green) while tracking particles. (b) (top) A 3D orbital scan path drives the laser (blue) along two circles at two different locations along the optical axis. Photons collected along the scan are used to estimate the position of the particle (green). (bottom) A sketch of the estimation result (in the plane). The black line is the path of the laser, the green circle is the true position of the particle, the black star is the estimated position, and the yellow ellipse represents the uncertainty in position estimation. (c) (top) A realization of single photon detection events as a function of the (bottom) orbital scan where the $x$ trajectory of the laser is shown in black and the true trajectory of the particle in green. (top) The rate of the photon detection events increases when the center of the laser is positioned closer to the particle.

The ultimate goal of RT-FD-SPT experiments is the determination of the state and dynamics of individual particles as they move in their native environments. As these microscopes only measure photon arrivals, the desired details must be estimated based on the acquired data. This places estimation at the core of RT-FD-SPT microscopes, and highlights the need for detected photons to be maximally informative towards the experimental aim. A useful tool in analyzing the information available in the measurements is the Fisher Information Matrix (FIM). In addition to describing the information available, its inverse, called the Cramer-Rao lower bound (CRLB), measures the best possible uncertainty in estimation [23]. In the context of SPT, Fisher information has been used as a tool for analyzing performance [24–26], for designing the point spread function, [27], for designing the system illumination [11,28,29], and for creating a controller [30,31]. Each of these works focuses on a single aspect of the general SPT problem. When viewed together, however, a general stochastic framework emerges for enabling the optimization of a microscope with respect to the underlying estimation goals in the presence of noise and uncertainty.

In this paper, we pose the information optimization framework as a tool to optimize the performance of RT-FD-SPT microscopes and demonstrate its application in the context of localizing a fluorescently labeled particle in three dimensions with a minimum localization uncertainty. Our approach extends the work of Gallatin and Berglund [28] and of our own prior work [30] in at least two significant ways: we derive the information optimal measurement locations when background and detector noise are present, showing how these terms affect the optimal measurement locations for localization, and we apply this framework to determine the optimal parameter defining the configurations of three existing RT-FD-SPT techniques, one based on a six-point measurement constellation [32], one based on a four-point measurement constellation [33], and one based on an orbital scheme [34].

2. Information optimal framework

When viewed through the lens of Fisher information, the detected signal in RT-FD-SPT (the photon counts) is described as a random process carrying information about an underlying collection of parameters, denoted as the vector variable $\theta$ , to be estimated from the measurements [35]. The information available in a set of observations $Y$ is given by the FIM, denoted by $F$ and defined by

$$F=\mathbb{E}\left[{\left(\frac{\mathrm{\partial }\mathcal{L}(Y|\theta )}{\mathrm{\partial }\theta }\right)}^{\prime }\left(\frac{\mathrm{\partial }\mathcal{L}(Y|\theta )}{\mathrm{\partial }\theta }\right)\right],$$ (1)

where $\mathcal{L}(Y|\theta )$ is the log-likelihood of measuring the set of observations conditioned on the set of parameters we want to infer, the notation ′ indicates the transpose operation, and $\mathbb{E}[]$ is the expectation taken with respect to the likelihood of measuring $Y$ given $\theta$ , $p(Y|\theta )$ .

The information optimal framework seeks to design the measurements so as to maximally inform our knowledge of the unknown parameter $\theta$ and therefore assumes the matrix depends on some user-defined choices; we denote these as $u$ so that $F=F(u)$ . We define the notion of “best” by converting the FIM to a scalar cost function. In addition, the measurement process may be subject to a variety of constraints, such as imposing a maximum excitation intensity or maximum light dosage so as to prevent photodamage to the sample, or actuator limits on how fast or how far the excitation laser can be moved. This leads us to the generic information optimal framework given by the optimization problem

$$\begin{array}{c}\underset{u}{max}\Vert F(u)\Vert \text{subject to }g(u)=0,\end{array}$$ (2)

where $g(\cdot )$ is a function defining the constraints on the problem and $\Vert \cdot \Vert$ is a given norm. The inverse of the FIM gives the Cramer-Rao lower bound that defines the best possible variance for any unbiased estimator [23]. Maximizing the FIM, then, leads to the best theoretical precision. (Of course, the actual achieved precision depends on the estimator used.)

There are many possible choices of the norm $\Vert \cdot \Vert$ to convert $F$ into a scalar cost. In the theory of optimal experimental design [36], one typically selects from the family of matrix means,

$$\Vert F{\Vert }_p={\left[\frac{1}{d}Tr\left(F^p\right)\right]}^{\frac{1}{p}}$$ (3)

where $p$ determines the specific norm and $d$ is the number of parameters being estimated. Four common choices are the so-called alphabetic criteria denoted as $T$ ( $p=1$ ), $D$ ( $p=0$ ), $A$ ( $p=-1$ ), and $E$ ( $p=-\mathrm{\infty }$ ). $T$ corresponds to the trace of the matrix, $D$ to its determinant, $A$ to the trace of the inverse, and $E$ to the maximum eigenvalue of the inverse. (Note that when using $A$ and $E$ , the $max$ in (2) should be replaced by $min$ .) While all of these metrics can be useful, throughout this work we choose to focus on $T$ -optimality as it is linear, local, and in general the most mathematically tractable. As we will see, this choice leads to some ambiguity as only the total trace is minimized, not each individual term. This can be overcome by imposing additional constraints such as requiring equality of each of the diagonal terms. Additionally, the trace considers only the diagonal elements, overlooking any correlation between the parameters appearing in the off-diagonal elements; The $D$ or $E$ metrics would avoid this, although at the cost of a more complicated optimization problem.

For the purposes of concreteness, for the remainder of this paper we assume the system is based on a laser scanning microscope such as a confocal or two-photon microscope. We denote the position of the center of the excitation laser as $r_l$ , the position of the particle as $r_p$ , and the relative position of the two as $ϵ$ ,

$$r_l={\left[\begin{array}{ccc}{x}_l& y_l& z_l\end{array}\right]}^{\prime },r_p={\left[\begin{array}{ccc}{x}_p& y_p& z_p\end{array}\right]}^{\prime },ϵ=r_p-r_l.$$ (4)

Note that because the location of the particle is unknown, the actual value of $ϵ$ is also unknown.

In the context of RT-FD-SPT, the user defined choices that influence the FIM include the use of structured illumination (using, e.g., a donut style beam as in [11]), structured detection (using, e.g., engineered point spread functions as in [2]), and controller design (see, e.g., [10]). Photon detection in a fluorescence microscope is a spatio-temporal process where the rate of detection is given by a Poisson distribution [24,28]. The corresponding FIM is then given by

$$F(u)={\int }_0^T\frac{1}{\mathrm{\Lambda }(u)+\beta }{\left(\frac{\mathrm{\partial }\mathrm{\Lambda }(u)}{\mathrm{\partial }\theta }\right)}^{\prime }\left(\frac{\mathrm{\partial }\mathrm{\Lambda }(u)}{\mathrm{\partial }\theta }\right)\mathrm{d}t,$$ (5)

where $\mathrm{\Lambda }(u)$ is the point spread function of the instrument, $\beta$ is a combination of background and dark detector rates under the assumption that these are Poisson distributed, and $T$ is the measurement time. Note that more sophisticated models can be derived that account for different noise characteristics of detectors. Our focus in this work, however, is on RT-FD-SPT where detectors are typically single photon counting modules for which this simple noise model is often a good approximation. In these microscopes, the rate $\mathrm{\Lambda }$ is well-approximated by a three dimensional Gaussian function [37,38], yielding a model for the detected photon rate

$$\mathrm{\Lambda }(ϵ)={\mathrm{\Lambda }}_o{e}^{-\frac{1}{2}ϵ\prime {\mathrm{\Sigma }}^{-1}ϵ},$$ (6)

where

$$\begin{array}{c}\mathrm{\Sigma }=\left[\begin{array}{ccc}\frac{{\omega }_x^2}{4}& 0& 0\\ 0& \frac{{\omega }_y^2}{4}& 0\\ 0& 0& \frac{{\omega }_z^2}{4}\end{array}\right].\end{array}$$ (7)

Here $({\omega }_x,{\omega }_y,{\omega }_z)$ are the beam diameters in the $(x,y,z)$ directions respectively. Inserting this model into Eq. (5) and assuming the unknown parameters are

$$\theta ={\left[\begin{array}{cccccc}{r}_p^{\prime }& {\mathrm{\Lambda }}_o& {\omega }_x& {\omega }_y& {\omega }_z& \beta \end{array}\right]}^{\prime },$$

leads to the FIM

$$F={\int }_0^T\frac{{\mathrm{\Lambda }}^2}{\mathrm{\Lambda }+\beta }\left[\begin{array}{cccccc}{\mathrm{\Sigma }}^{-1}ϵϵ\prime {\mathrm{\Sigma }}^{-1}& -\frac{1}{{\mathrm{\Lambda }}_0}{\mathrm{\Sigma }}^{-1}ϵ& \frac{4{ϵ}_x^2}{{\omega }_x^3}{\mathrm{\Sigma }}^{-1}ϵ& \frac{4{ϵ}_y^2}{{\omega }_y^3}{\mathrm{\Sigma }}^{-1}ϵ& \frac{4{ϵ}_z^2}{{\omega }_z^3}{\mathrm{\Sigma }}^{-1}ϵ& -\frac{ϵ\prime {\mathrm{\Sigma }}^{-1}}{\mathrm{\Lambda }}\\ -\frac{1}{{\mathrm{\Lambda }}_0}ϵ\prime {\mathrm{\Sigma }}^{-1}& \frac{1}{{\mathrm{\Lambda }}_0^2}& -\frac{4{ϵ}_x^2}{{\mathrm{\Lambda }}_0{\omega }_x^3}& -\frac{4{ϵ}_y^2}{{\mathrm{\Lambda }}_0{\omega }_y^3}& -\frac{4{ϵ}_z^2}{{\mathrm{\Lambda }}_0{\omega }_z^3}& \frac{1}{{\mathrm{\Lambda }}_0\mathrm{\Lambda }}\\ \frac{4{ϵ}_x^2}{{\omega }_x^3}ϵ\prime {\mathrm{\Sigma }}^{-1}& -\frac{4{ϵ}_x^2}{{\mathrm{\Lambda }}_0{\omega }_x^3}& \frac{16{ϵ}_x^4}{{\omega }_x^6}& \frac{16{ϵ}_x^2{ϵ}_y^2}{{\omega }_x^3{\omega }_y^3}& \frac{16{ϵ}_x^2{ϵ}_z^2}{{\omega }_x^3{\omega }_z^3}& -\frac{4{ϵ}_x^2}{{\omega }_x^3\mathrm{\Lambda }}\\ \frac{4{ϵ}_y^2}{{\omega }_y^3}ϵ\prime {\mathrm{\Sigma }}^{-1}& -\frac{4{ϵ}_y^2}{{\mathrm{\Lambda }}_0{\omega }_y^3}& \frac{16{ϵ}_x^2{ϵ}_y^2}{{\omega }_x^3{\omega }_y^3}& \frac{16{ϵ}_y^4}{{\omega }_y^6}& \frac{16{ϵ}_y^2{ϵ}_z^2}{{\omega }_y^3{\omega }_z^3}& -\frac{4{ϵ}_y^2}{{\omega }_y^3\mathrm{\Lambda }}\\ \frac{4{ϵ}_z^2}{{\omega }_z^3}ϵ\prime {\mathrm{\Sigma }}^{-1}& -\frac{4{ϵ}_z^2}{{\mathrm{\Lambda }}_0{\omega }_z^3}& \frac{16{ϵ}_x^2{ϵ}_z^2}{{\omega }_x^3{\omega }_z^3}& \frac{16{ϵ}_y^2{ϵ}_z^2}{{\omega }_y^3{\omega }_z^3}& \frac{16{ϵ}_z^4}{{\omega }_z^6}& -\frac{4{ϵ}_z^2}{{\omega }_z^3\mathrm{\Lambda }}\\ -\frac{ϵ\prime {\mathrm{\Sigma }}^{-1}}{\mathrm{\Lambda }}& \frac{1}{{\mathrm{\Lambda }}_0\mathrm{\Lambda }}& -\frac{4{ϵ}_x^2}{{\omega }_x^3\mathrm{\Lambda }}& -\frac{4{ϵ}_y^2}{{\omega }_y^3\mathrm{\Lambda }}& -\frac{4{ϵ}_z^2}{{\omega }_z^3\mathrm{\Lambda }}& \frac{1}{{\mathrm{\Lambda }}^2}\end{array}\right]\mathrm{d}t.$$ (8)

Each of the diagonal elements in Eq. (8) represents the Fisher information for the corresponding parameter. If we specify the user defined signal $u$ to be a fixed position of the laser (and consequently, a fixed value of $ϵ$ ), we can use the FIM to determine the amount of information available to estimate each of the parameters from data collected with the laser at that position. In Fig. 2 we show maps of the Fisher information as a function of the position of the laser in the focal plane for $x$ , the position of the particle along the $x-$ axis, ${\mathrm{\Lambda }}_0$ , the peak detected photon rate, ${\omega }_x$ , the beam width in the $x-$ direction, and $\beta$ , the noise rate. We also show the FIM for the corresponding variable along the red scan line on the map, together with the function $\mathrm{\Lambda }$ along that same scan for reference. One of the main features of this set of plots is that the best location to place the excitation beam is different depending on which parameter you want to estimate. For example, to maximize information about the peak photon rate ${\mathrm{\Lambda }}_0$ , the laser should be placed directly on top of the particle; however, this is also the point of minimum information for all other parameters. This strongly suggests that multiple measurements should be taken to balance the information acquired over the parameters of interest.

Fig. 2.

Example Fisher information as a function of the relative position of the excitation laser and particle for the (a) $x$ -position, (b) peak photon rate ${\mathrm{\Lambda }}_o$ , (c) beam diameter ${\omega }_x$ , and (d) noise parameter $\beta$ . Also shown is the FIM along the red line scan, with the Gaussian laser shape superimposed for reference. Dimensions were normalized by the beam widths. The peak intensity was set to 1 and the noise rate to 0.1.

It is important to note that there is a finite amount of information available in a given set of measurements; using that data to estimate more unknowns necessarily means reduced precision in those estimates. Thus it is vital that one determine what can be reasonably known (for example, approximating the background rate as constant and measuring that separately) to remove those terms from the list of parameters to be estimated. Note also that these measurements are typically of a moving particle. Since photons are acquired over time, that motion affects the data (commonly known as motion blur). While this motion can be incorporated into the models and from there into FIM, for simplicity we will make the common assumption that the particle is fixed during the measurement process. In practice, this implies that the particle is moving slowly with respect to that time scale of the measurements.

3. Information optimal measurement points

To better understand how to deploy the information optimal methodology, in this section, we consider how to select a (fixed) measurement point to solve the optimization problem in Eq. (2). Suppose, for example, that one is interested in determining a single parameter of the tracked particle. Our optimization problem then becomes

$$\underset{x_l}{max}{[F]}_{ii},$$ (9)

where ${[\cdot ]}_{ii}$ denotes the $i^{th}$ diagonal entry in the FIM, corresponding to the parameter of interest. This is then a fairly straightforward optimization problem over the three-dimensional position of the laser. If the parameter of interest is the $x-$ position of the particle, this optimization problem is to find the maximum of the FIM shown in Fig. 2(a). Taking the derivative of ${[F]}_{11}$ with respect to ${ϵ}_x$ and setting it to zero yields

$${ϵ}_x^2(2\mathrm{\Lambda }+4\beta )-{\omega }_x^2(\mathrm{\Lambda }+\beta )=0.$$ (10)

Keeping in mind that $\mathrm{\Lambda }=\mathrm{\Lambda }(ϵ)$ , this is a nonlinear optimization problem. In the limit, as the Signal-to-Noise Ratio (SNR) (defined here as the ratio of ${\mathrm{\Lambda }}_o$ to $\beta$ ) goes to infinity ( $\beta \to 0$ ), the solution is

$${ϵ}_x^{\mathrm{\infty }}=\pm \frac{{\omega }_x}{\sqrt{2}},$$ (11)

matching what has been previously found [28]. In the opposite limit, as the the SNR goes to zero ( $\beta \to \mathrm{\infty }$ ), we find

$${ϵ}_x^0=\pm \frac{{\omega }_x}{2}.$$ (12)

The results for intermediate values of the SNR are shown in Fig. 3, with panel (a) showing the FIM for the $x-$ position for a range of points along the $x-$ axis and at different SNRs (with the Gaussian excitation beam once again shown for reference), while panel (b) shows the location of the optimal distance for the (fixed) position of the excitation laser as a function of the SNR. These results match the intuition that as the background noise increases, the excitation should be moved closer to the particle to increase the signal level, while also matching the intuitive result that an increased background or noise rate implies lower information (for a fixed measurement time). Since in the case of a fixed measurement location the FIM is proportional to the measurement time, this also implies that to get an equivalent amount of information in the presence of higher noise requires measuring for longer times. For example for a fixed measurement location, from (5) we see that to get the equivalent amount of information from a measurement with noise rate $\beta$ as is in a measurement with $\beta =0$ , we would need to measure for a duration given by

$$T_{\beta }=T\frac{\mathrm{\Lambda }+\beta }{\mathrm{\Lambda }},$$

where $T$ is the duration of the noise-free measurement.

Fig. 3.

Effect of background noise on FIM. (a) FIM for particle $x$ position as a function of the relative position of the excitation laser and particle at different SNR values. (b) Optimal distance of the excitation laser along the $x$ -axis as a function of SNR.

If instead one is interested in the $y$ or $z$ positions, the derivation is the same but with ${ϵ}_x$ replaced by ${ϵ}_y$ ( ${ϵ}_z$ ) and ${\omega }_x$ by ${\omega }_y$ ( ${\omega }_z$ ) and in general we can write

$${ϵ}_{xyz}^{\text{opt}}=\pm \alpha {\omega }_{xyz},$$ (13)

where $\alpha \in [1/2,1/\sqrt{2}]$ is a scale factor that depends on the SNR. Note that the value of $\alpha$ is the same for each coordinate direction.

As previously discussed, the optimal single measurement location is different for different parameters. When interested in multiple parameters, then, one can introduce an additional weighting matrix to balance the relative importance of the different terms or to account for different units in the parameters of interest. For example, if one is interested in localizing the particle in three dimensions, the optimization problem in (2) becomes

$$\underset{x_l}{max}\frac{{\mathrm{\Lambda }}^{2}T}{\mathrm{\Lambda }+\beta }\text{Tr}[W{\mathrm{\Sigma }}^{-1}ϵ\prime ϵ{\mathrm{\Sigma }}^{-1}W].$$ (14)

Here $\text{Tr}[\cdot ]$ is the trace operator and $W$ is a diagonal matrix of weighting terms. Note that the need for such a weighting matrix can be avoided using a different metric such as the determinant ( $p=0$ in (3)) or the maximum eigenvalue of the inverse ( $p=-\mathrm{\infty }$ ). It can be shown that the set of solutions to (14) is an ellipsoidal surface centered on the position of the particle with radii determined by the noise, the weighting terms, and the beam widths. We refer to this surface as the information optimal set. From the point of view of the information optimal problem, all of these locations are equivalent. While ambiguous, this also provides the freedom to design a trajectory in this set so as to satisfy some secondary criteria such as, for example, balancing the information obtained about each of the three position parameters. In the two dimensional setting of estimating the particle location in the plane, for example, the information optimal set is an ellipse and moving along an integer number of half-orbits ensures that the CRLB for the $x$ and $y$ location of the particle are equal [30].

Note that this analysis assumes a fixed measurement time $T$ and in essence we are thinking of information received per unit of time. One can also formulate the optimization problem for a fixed number of total photons received (with the time being free), and thus in terms of information per photon. For the problem of localization in a single axis in the absence of background, this leads to a solution of placing the excitation laser infinitely far away. While each photon would then carry maximal information about where the particle is, we would never actually receive any of those photons and thus additional constraints are needed to ensure the problem is well-formulated. See [11] for some work along these lines.

4. Information optimal RT-FD-SPT

Most RT-FD-SPT methods rely on online estimation of particle location to achieve tracking of a moving particle, and even when using an algorithm that does not rely on positional information, determining the path of the particle is typically a key piece of information. In this section, we demonstrate how to use the information optimal approach to optimize the controller parameters of three existing RT-FD-SPT methods so that the data they collect maximally (in terms of the $T$ -criterion in (3)) inform the localization problem. While our approach is not limited to these three approaches, they are representative of multiple techniques in the literature and are fairly straightforward to analyze. Throughout, we make the standard simplifying assumption that the particle is fixed. Practically, this implies that the motion of the particle is slow relative to the rate at which measurements are collected.

Note that while we focus on optimizing user-selectable parameters in these algorithms as these are likely the easiest in practice to adjust, one could also optimize beam parameters (such as beam width or shape).

4.1. Six-point measurement constellation

One of the authors previously developed an RT-FD-SPT algorithm that uses a six-point measurement constellation (illustrated in Fig. 4(a)) [16,32]. The essential idea of this method is to move the excitation source through these six points, dwelling for some time at each to collect photons, use those six measurements to estimate the particle location, and then use a feedback controller to drive the (estimated) error between the center of the constellation and the location of the particle to zero. Under the assumption that the constellation points are symmetric about each of the axes, the free parameters that define the constellation are the three distances $(r_x,r_y,r_z)$ and the three times $(t_x,t_y,t_z)$ to collect measurements at each pair, equally divided between the two locations on the corresponding axis. Note that while the distances to the two measurement locations on an axis, and the time spent on each, could be different, the symmetry of the excitation beam implies these values should also be symmetric.

Fig. 4.

Six point measurement constellation from [32]. (a) 3D view. (b) Projection onto $x-y$ plane. (c) Projection onto $x-z$ plane.

If we impose the constraint that the Fisher information with respect to each of the three coordinate axes should be the same, we arrive at the information optimization problem

$$\begin{array}{ll}\underset{(r_x,r_y,r_z,t_x,t_y,t_z)}{max}& \sum _{j=1}^6{\int }_{t_{j-1}}^{t_j}\frac{{\mathrm{\Lambda }}^2({ϵ}_j)}{\mathrm{\Lambda }({ϵ}_j)+\beta }\left({\mathrm{\Sigma }}^{-1}{ϵ}_j{ϵ}_j^{\prime }{\mathrm{\Sigma }}^{-1}\right)\mathrm{d}t\\ \hfill \text{subj. to }& {[F]}_{11}-{[F]}_{22}=0,{[F]}_{22}-{[F]}_{33}=0,\\ & t_1-t_0=t_2-t_1=\frac{t_x}{2},t_3-t_2=t_4-t_3=\frac{t_y}{2},t_5-t_4=t_6-t_5=\frac{t_z}{2},\\ & t_x+t_y+t_z=T,\end{array}$$ (15)

where $T$ is the total measurement time and we have assumed (without loss of generality) that the measurement order is the points along the $x$ -axis, then the $y$ -axis, and then the $z$ -axis. With this assumption and defining the origin of the coordinate system to be at the particle ( $x_p=0$ ), we have

$$\begin{array}{c}\left[\begin{array}{cccc}{ϵ}_1& {ϵ}_2& \cdots & {ϵ}_6\end{array}\right]=\left[\begin{array}{cccccc}{r}_x& -r_x& 0& 0& 0& 0,\\ 0& 0& r_y& -r_y& 0& 0,\\ 0& 0& 0& 0& r_z& -r_z.\end{array}\right].\end{array}$$ (16)

From the analysis in Sec. 3, we know that placing the measurement locations on the information optimal set ensures the maximum Fisher information from each measurement for any given length of time. To determine the dwell time, we insert these locations, given by Eq. (13), into our constraints and then solve to find

$$\begin{array}{c}{t}_x=\frac{T{\omega }_x^2}{{\omega }_x^2+{\omega }_y^2+{\omega }_z^2},\end{array}$$ (17a)

$$\begin{array}{c}{t}_y=\frac{T{\omega }_y^2}{{\omega }_x^2+{\omega }_y^2+{\omega }_z^2},\end{array}$$ (17b)

$$\begin{array}{c}{t}_z=\frac{T{\omega }_z^2}{{\omega }_x^2+{\omega }_y^2+{\omega }_z^2}.\end{array}$$ (17c)

Note that the optimal times do not depend on the noise value $\beta$ , though the optimal locations and the amount of information acquired does. This result simply says that the optimal share of time to spend at each measurement location for a given fixed total time $T$ is independent of the noise.

Figure 5 shows the Fisher information for estimation of the $x$ , $y$ , and $z$ location of the particle under an optimal six-point measurement constellation as a function of the true position of the particle relative to the center of the constellation in the (top row) $x-y$ plane and (bottom row) $x-z$ plane. The beam width parameters were set to ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, the peak intensity is set to ${\mathrm{\Lambda }}_0=1$ , and the background rate is set to $\beta =0$ . The units of the Fisher information are thus ${\mathrm{units}}^{-2}$ . Note the different scaling in the $z$ FIM relative to the $x$ and $y$ ; the optimization ensures only that when the particle is at the center, the Fisher information values for each direction are balanced. This implies that the values at the center of each of the FIM plots in Fig. 5 are the same. Interestingly, as the particle moves away from the center in the plane, there can be increased information for position estimation in the $x$ and $y$ axes depending on where the particle is. This is seen by the increased FIM values for the $x$ and $y$ localization away from the center of the plots. This arises from the fact that two measurement points above and below the plane contribute no information to the lateral localization if the particle is in the center, but do contribute such information if the particle is away from the center axis. However, the FIM values for $x$ , $y$ , and $z$ are no longer balanced.

Fig. 5.

FIM for the optimal six point measurement constellation for the (a) $x$ -, (b) $y$ -, and (c)  $z$ -location of the particle as a function of the position of the particle with respect to the center of the constellation in the (top) $x-y$ and (bottom) ( $x-z$ ) planes. White circles indicate the measurement locations.

Note that the optimal configuration for this six-point constellation was previously considered in [39] under the assumption of a large signal level. That work sought to minimize the variance of a particular localization algorithm (the fluoroBancroft algorithm [40]) under the assumption of equal measurement times, and found optimal positions which match the information optimal set defined in Sec. 3. Our present work is more general in that it allows for optimization of the information for multiple parameters, not just particle location, and is independent of any particular estimator.

In Fig. 6, we illustrate the impact of the choice of parameters defining the measurement pattern. In each figure, for each choice of parameter value, we performed 1000 Monte Carlo simulations of the measurement process, determining the number of photons collected at each measurement location by sampling from a Poisson process with a rate defined by the measurement location with the optical parameters set to ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, ${\mathrm{\Lambda }}_o=100$ , and $\beta =0$ . The particle was placed at the center of the constellation. Figure 6(a) demonstrates the impact of varying the scale of the entire constellation with the Fisher information for the $x-$ position shown in blue and the number of photons collected at the first measurement location (at $(r_x,0,0)$ ) in red. For each, the solid line is the mean value over the Monte Carlo runs and the shaded region is $\pm$ one standard deviation (bounded below by zero). At a scale of 0, all four points are at the origin, maximizing the number of photons collected but minimizing the Fisher information. As the constellation expands, the photon counts drop while the information increases until the peak at the information optimal location, after which it drops.

Fig. 6.

FIM for the optimal six point measurement constellation for a particle at the center of the pattern. 1000 Monte Carlo runs with ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, ${\mathrm{\Lambda }}_o=100$ . (a) (blue) Fisher information for the $x-$ position and (red) measured photons at the measurement location on the positive $x-$ axis as a function of the scale of the constellation relative to the optimal size. (b) Fisher information for the (blue) $x-$ and (red) $z-$ position of the particle as a function of the fraction of time spent at the four measurement locations in the optical plane.

Figure 6(b) shows the impact of changing the fraction of time spent on measuring at the four points in the plane with the information for estimating the $x-$ position shown in blue and for the $z-$ position in magenta. As expected, spending more time on the two measurement locations on the $z-$ axis increases the information available for estimating the axial position, at the cost of reducing the information available for lateral position estimation. The black vertical line indicates the optimal location where the information for each axis is balanced.

4.2. Four-point measurement constellation (TSUNAMI)

In [33], an RT-FD-SPT algorithm using pulsed interleaved excitation was introduced. Named Tracking Single particles Using Nonlinear and Multiplexed Illumination (TSUNAMI), the method uses passive pulse splitters with carefully determined delays to achieve spatiotemporally multiplexed two-photon excitation at four different locations arranged in a tetrahedral pattern. This pattern, used also in [41,42] is illustrated in Fig. 7.

Fig. 7.

Four point measurement constellation from [33]. (a) 3D view. (b) Projection onto $x-y$ plane. (c) Projection onto $x-z$ plane.

Because of the specific design of this approach, it is easiest to set the measurement times in each location to all be equal to $T/4$ and to optimize the pattern to maximize the Fisher information subject to the constraint that the information for each axis is the same. The pattern is defined by two parameters, the distance along the $x$ or $y$ axis to the measurement points ( $r_{xy}$ ), and the distance along the optical axis ( $r_z$ ) to those points. This leads to the optimization problem

$$\begin{array}{cc}\underset{(r_{xy},r_z)}{max}& \sum _{j=1}^4{\int }_{t_{j-1}}^{t_j}\frac{{\mathrm{\Lambda }}^2({ϵ}_j)}{\mathrm{\Lambda }({ϵ}_j)+\beta }\left({\mathrm{\Sigma }}^{-1}{ϵ}_j{ϵ}_j^{\prime }{\mathrm{\Sigma }}^{-1}\right)\mathrm{d}t\\ \text{subj. to }& {[F]}_{11}-{[F]}_{22}=0,{[F]}_{22}-{[F]}_{33}=0,\\ & t_j-t_{j-1}=\frac{T}{4},j=1,2,3,4,\end{array}$$ (18)

with

$$\begin{array}{c}\left[\begin{array}{cccc}{ϵ}_1& {ϵ}_2& {ϵ}_3& {ϵ}_4\end{array}\right]=\left[\begin{array}{cccc}{r}_{xy}& -r_{xy}& 0& 0\\ 0& 0& r_{xy}& -r_{xy}\\ -r_z& -r_z& r_z& -r_z\end{array}\right],\end{array}$$ (19)

where once again we have defined the center of the coordinate system to be at the particle location.

Inserting these positions into the constraint that ${[F]}_{11}={[F]}_{22}$ reveals that either the excitation beam must be symmetric in the plane ( ${\omega }_x={\omega }_y$ ) or the distances to the measurement points along the $x$ and $y$ axes must be different. For the purposes of simplicity, we assume the symmetric case. Applying the second constraint of ${[F]}_{11}={[F]}_{33}$ leads to

$$r_z=\frac{{\omega }_z^2}{\sqrt{2}{\omega }_x^2}{r}_{xy}.$$ (20)

Putting this result into the optimization function, setting its derivative with respect to $r_{xy}$ to zero, and solving, yields

$$r_{xy}=\frac{{\omega }_x^2}{\sqrt{2{\omega }_x^2+{\sigma }_z^2}},r_z=\frac{{\omega }_z^2}{\sqrt{2}\sqrt{2{\omega }_x^2+{\omega }_z^2}}.$$ (21)

Figure 8 shows the Fisher information for estimation of the $x$ , $y$ , and $z$ location of the particle under an optimal four-point configuration as a function of the true position of the particle relative to the center of the constellation in the (top row) $x-y$ plane and (bottom row) $x-z$ plane. As before, the parameters were set to ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, ${\mathrm{\Lambda }}_0=1$ , $\beta =0$ . While the FIM is similar to the 6-point configuration, there is less symmetry overall, particularly as the particle moves in the $x-z$ plane. Note that given the same total measurement time, the information available for estimation for a particle at the center of the pattern is the same in the 4-point as in the 6-point.

Fig. 8.

FIM for the optimal four point measurement constellation for the (a) $x$ -, (b) $y$ - and (c) $z$ -location of the particle as a function of the position of the particle with respect to the center of the constellation in the (top) $x-y$ and (bottom) ( $x-z$ ) planes. White circles indicate the measurement locations.

As with the 6-point constellation, we performed 1000 Monte Carlo simulations of the measurement process for each parameter value, sweeping over the scale of the constellation (Fig. 9(a)) and the value of $r_z$ , keeping $r_{xy}$ constant (Fig. 9(b)). Once again, when the scale of the constellation is zero and all the measurement points are at the origin, the number of photons collected is maximized but the information available in those photons is zero. Increasing the scale to the optimal decreases the photon counts but increases the information to a peak at the optimal size, before the information falls again.

Fig. 9.

Fisher information for the four point constellation over 1000 Monte Carlo runs with ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, and ${\mathrm{\Lambda }}_o=100$ . (a) (blue) Fisher information for the $x-$ position and (red) number of measured photons at measurement location $(r_{xy},0,r_z)$ as a function of the scale of the constellation relative to the optimal size. (b) (blue) Fisher information for the (left) $x-$ position and (right) $z-$ position and (red) number of measured photons at the measurement location $(r_{xy},0,-r_z)$ (both plots).

The behavior when changing only $r_z$ is somewhat different. The two plots in Fig. 9(b) show the (left, blue) information for estimating the $x-$ position and (right, blue) for the $z-$ position, as well as the number of photons acquired at the $(r_{xy},0,-r_x)$ location (red, both plots). At $r_z=0$ , the four measurements are in the horizontal plane of the particle, leading to a maximal number of photons for this pattern and a maximal information available for estimating the $x-$ position. However there is no information available for estimating the $z-$ position. As the measurement locations are moved away from the plane, the photon counts decrease and the information for $x-$ estimation decreases, shifting into information for $z-$ estimation. The information for $x-$ and $z-$ estimation become equal at the optimal radius. Interestingly, the information available for $z$ continues to increase past the optimal, though at the cost of estimation in $x$ (and, due to symmetry, $y$ ). This is because the optimal measurement locations under the constraint of equal times is not on the information optimal set found in Sec. 3.

4.3. Orbital tracking

One of the most successful and most used methods of RT-FD-SPT is that of 3D orbital tracking [34]. This scanning method utilizes two orbits centered on the (expected) position of the particle, one in a plane below the particle, and one in a plane above. This pattern is illustrated in Fig. 10. The free parameters are the radius of the orbit, $r$ , the frequency of orbiting (or, equivalently, the number of orbits to complete in a given time $T$ ), ${\omega }_1$ , and the position of the orbits along the vertical axis $\pm r_z$ . This leads to the optimization problem

$$\begin{array}{cc}\underset{{\omega }_1,(r_{xy},r_z)}{max}& \sum _{j=1}^2{\int }_{t_{j-1}}^{t_j}\frac{{\mathrm{\Lambda }}^2({ϵ}_j)}{\mathrm{\Lambda }({ϵ}_j)+\beta }\left({\mathrm{\Sigma }}^{-1}{ϵ}_j{ϵ}_j^{\prime }{\mathrm{\Sigma }}^{-1}\right)\mathrm{d}t\\ \text{subj. to }& {[F]}_{11}-{[F]}_{22}=0,{[F]}_{22}-{[F]}_{33}=0,\\ & t_j-t_{j-1}=\frac{T}{2},j=1,2,\end{array}$$ (22)

with

$$\begin{array}{c}\left[\begin{array}{cc}{ϵ}_1(t)& {ϵ}_2(t)\end{array}\right]=\left[\begin{array}{cc}r\cos \left({\omega }_{1}t\right)& r\cos \left({\omega }_{1}t\right)\\ r\sin \left({\omega }_{1}t\right)& r\sin \left({\omega }_{1}t\right)\\ -r_z& r_z\end{array}\right].\end{array}$$ (23)

Fig. 10.

Orbital pattern from [34]. (a) 3D view. (b) Projection onto $x-y$ plane. (c) Projection onto $x-z$ plane.

As before, the origin of the coordinate system is defined to be at the particle position.

Similar to the four-point constellation, the constraint that ${[F]}_{11}={[F]}_{22}$ reveals that either the beam must be symmetric in the plane ( ${\omega }_x={\omega }_y$ ) or the orbital pattern must be altered to allow for an ellipse with major and minor axes aligned to those of the beam. For the purposes of simplicity, we assume the symmetric constraint. With this assumption, the same constraint yields

$$\begin{array}{ll}0& ={[F]}_{11}-{[F]}_{22}\\ & =\frac{32\overline{\mathrm{\Lambda }}{r}^2}{{\omega }_x^4}{\int }_0^{T/2}({\cos }^2({\omega }_{1}t)-{\sin }^2({\omega }_{1}t))dt=\frac{32\overline{\mathrm{\Lambda }}{r}^2}{{\omega }_x^4}{\int }_0^{T/2}\cos (2{\omega }_{1}t)dt.\end{array}$$

From this we conclude that ${\omega }_1=2k\pi /T$ , that is, the excitation beam should be scanned along an integral number of half circles. Since in practice a full orbit is typically used, we set ${\omega }_1=4\pi /T$ .

With these choices, applying the second constraint of ${[F]}_{11}={[F]}_{33}$ leads to

$$r_z=\frac{{\omega }_z^2}{\sqrt{2}{\omega }_x^2}r.$$ (24)

Inserting this into the optimization function, settings its derivative with respect to $r$ to zero and solving yields

$$r=\frac{{\omega }_x^2}{\sqrt{2{\omega }_x^2+{\sigma }_z^2}},r_z=\frac{{\omega }_z^2}{\sqrt{2}\sqrt{2{\sigma }_x^2+{\sigma }_z^2}}.$$ (25)

Note that these match the parameters of the four point constellation, as expected given the symmetry of the two patterns.

Figure 11 shows the Fisher information for estimation of the $x$ , $y$ , and $z$ location of the particle under an optimal orbital pattern as a function of the true position of the particle relative to the center of the constellation in the (top row) $x-y$ plane and (bottom row) $x-z$ plane. Once again the parameters were set to ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, ${\mathrm{\Lambda }}_0=1$ , and $\beta =0$ . While the FIM in the $x-y$ plane is similar to both the 6-point and 4-point patterns, it is significantly different in the $x-z$ plane. The information for estimation for a particle at the center of the pattern is the same as for the other two sensing patterns; the differences come when the particle is off-center.

Fig. 11.

FIM for the optimal orbital measurement pattern for the (a) $x$ -, (b) $y$ -, and (c)  $z$ -location of the particle as a function of particle position with respect to the pattern center in the (top) $x-y$ and (bottom) $x-z$ planes. White lines indicate the orbital pattern.

To explore the impact of the parameters, we performed 1000 Monte Carlo simulations of the measurement process for a particle at the center of the pattern, sweeping over the radius (Fig. 12(a)) and the vertical placement of the orbits (Fig. 12(b)). The figures show the Fisher information for estimating the (blue) $x-$ and (red) $z-$ position of the particle. (Due to symmetry, the Fisher information for $y$ is the same as for $x$ with the particle at the origin.) With a radius of zero, there are two measurement points at $\pm r_z$ , leading to maximal information about the $z-$ position, but no information about the position in the plane. As the radius is expanded, the information flows out of the $z-$ direction and into the lateral estimation, reaching equality at the optimal radius. As the radius continues to increase, eventually the photon rate is so low that the total information available begins to drop.

Fig. 12.

Fisher information for the orbital constellation over 1000 Monte Carlo runs with ${\omega }_x={\omega }_y=1$ unit, ${\omega }_z=2$ units, and ${\mathrm{\Lambda }}_o=100$ . (a) Fisher information for the (blue) $x-$ position and (red) $z-$ position as a function of the orbital radius relative to the optimal size. (b) Fisher information for the (blue) $x-$ position and (red) $z-$ position as a function of the vertical location of the orbits relative to the optimal location.

Conversely, when $r_z$ is set to zero, all the information goes into estimating the lateral position with none available to estimate the $z-$ position. Increasing the separation of the two orbits transfers that information out of the lateral directions and into the vertical directions, reaching equality at the optimal value and eventually falling to zero as the orbits move far away from the particle.

5. Conclusion

The information optimal framework developed here allows designers of RT-FD-SPT microscopes to select the parameters defining their method of choice using a mathematically rigorous optimizing framework, ensuring that their measurements maximally inform their experimental aim. While we have illustrated this approach for estimating the particle location for three different RT-FD-SPT methods, the framework is applicable for other parameters of interest. The primary challenge is in developing the Fisher information for a parameter of interest. While the examples shown here are relatively straightforward and mathematically tractable, other cases may require numerical approaches (see, for example, [43] for a derivation of the FIM for estimating motion parameters such as diffusion coefficients in the wide-field SPT setting). These results show, however, that the choice of parameters in the RT-FD-SPT algorithm can have a significant impact on the quality of information acquired.

Funding

National Institute of General Medical Sciences10.13039/100000057 (5R01GM117039-02).

Disclosures

The authors declare no conflicts of interest.

Data availability

No data beyond simulations were generated or analyzed in the presented research. Code for simulations may be obtained from the authors upon reasonable request.