Real-Time Feedback-Driven Single-Particle Tracking: A Survey and Perspective

Bertus van Heerden; Nicholas A Vickers; Tjaart PJ Krüger; Sean B Andersson

doi:10.1002/smll.202107024

. Author manuscript; available in PMC: 2023 Jul 1.

Published in final edited form as: Small. 2022 Jun 27;18(29):e2107024. doi: 10.1002/smll.202107024

Real-Time Feedback-Driven Single-Particle Tracking: A Survey and Perspective

Bertus van Heerden ^1,², Nicholas A Vickers ³, Tjaart PJ Krüger ^1,², Sean B Andersson ^3,⁴

PMCID: PMC9308725 NIHMSID: NIHMS1819935 PMID: 35758534

Abstract

Real-time feedback-driven single-particle tracking (RT-FD-SPT) is a class of techniques in the field of single-particle tracking that uses feedback control to keep a particle of interest in a detection volume. These methods provide high spatiotemporal resolution on particle dynamics and allow for concurrent spectroscopic measurements. This review article begins with a survey of existing techniques and of applications where RT-FD-SPT has played an important role. We then systematically discuss each of the core components of RT-FD-SPT in order to develop an understanding of the trade-offs that must be made in algorithm design and to create a clear picture of the important differences, advantages, and drawbacks of existing approaches. These components are feedback tracking and control, ranging from simple proportional-integral-derivative control to advanced nonlinear techniques, estimation to determine particle location from the measured data, including both online and offline algorithms, and techniques for calibrating and characterizing different RT-FD-SPT methods. We then introduce a collection of metrics for RT-FD-SPT to help guide experimentalists in selecting a method for their particular application and to help reveal where there are gaps in the techniques that represent opportunities for further development. Finally, we conclude with a discussion on future perspectives in the field.

Keywords: single-particle tracking, single molecule spectroscopy, active feedback tracking

Graphical Abstract

graphic file with name nihms-1819935-f0001.jpg

An emerging approach to studying freely moving single particles and molecules is Real-Time Feedback-Driven Single-Particle Tracking (RT-FD-SPT) and RT-FD-SPT spectroscopy. This paradigm enables simultaneous tracking and spectroscopic measurements of particles in their native environments. This review surveys methods and applications of RT-FD-SPT, describes the main components of existing approaches and how each influences performance, and concludes with future perspectives.

1. Introduction

The ability to detect individual molecules and observe their dynamical behavior has revolutionized life sciences during the past three decades. State-of-the-art techniques provide nanometer-scale spatial resolution at a time resolution of 1 ms or better. An important single-molecule microscopy technique, known as single-particle tracking (SPT) [1, 2, 3, 4, 5, 6, 7, 8], allows direct access to the dynamics of individual freely diffusing molecules and particles in live cells. The conventional approach to SPT involves acquiring a sequence of images of fluorescent particles, determining their positions in each frame, and linking those into a trajectory for further analysis. Such image-based SPT can achieve excellent three-dimensional spatial resolution (10–20 nm) [9, 10, 11], but the time resolution is limited to 30–100 ms [10, 12, 11] due to the integration time needed to collect enough photons (ca. 500–10,000) [11, 9] to form an image.

Another single-molecule detection technique that has significantly advanced our understanding of the properties and dynamics of biomolecules is known as single-molecule spectroscopy (SMS) [13]. SMS is an umbrella term for various types of measurements on single molecules and molecular or nanoscale systems that involve photoexcitation of photoactive labels attached to the molecular systems or direct excitation of the molecules or particles. Fluorescence photons are typically detected and used to obtain information such as the emission brightness, spectra, and fluorescence lifetimes. The selectivity and sensitivity are significantly enhanced by isolating the biomolecules from their natural environment, and the information content is increased by a few orders of magnitude through immobilization of the molecules or confinement of their motion to a small subspace to allow sufficiently long observation times. Such modifications, however, significantly alter the setting in which the target molecules are being studied. An important challenge facing SMS, therefore, is the ability to measure biological molecules in their natural environment and for sufficiently long durations to obtain meaningful information but without compromising their structure and dynamics.

Real-time feedback-driven single-particle tracking (RT-FD-SPT) is a technique that offers a marked improvement in temporal resolution and an extended range– especially in the axial direction– compared to image-based SPT and additionally enables concurrent spectroscopic measurements to be made on the tracked particles. In most cases, image-based SPT does not allow concurrent SMS because the calculation of the particle location is too slow. In contrast, RT-FD-SPT allows SMS to be performed on freely moving biomolecules. This capability overcomes some of the major current limitations of standard SMS, allowing for long observation times without significant alteration of the biomolecular motion. RT-FD-SPT thus represents an important stepping stone towards realizing in vivo SMS.

Thus far, the development of RT-FD-SPT has been somewhat fractured, with different research groups coming up with independent solutions. This is due, at least in part, to its interdisciplinary nature, with approaches relying on techniques from physics, engineering, chemistry, and biology. There have thus far been only three reviews focusing solely on RT-FD-SPT [14, 15, 16]. While some reviews of image-based SPT have touched upon RT-FD-SPT methods [1, 5, 6, 8], they do not consider this topic in depth. Ref. [14] dates from 2008 and there have since been many developments. Ref. [15] focused almost exclusively on one specific technique (3D multi-resolution microscopy). The most recent review [16] described a variety of methods and compared their experimental performance but did not present a systematic analysis of RT-FD-SPT. Because the different approaches towards RT-FD-SPT have different strengths and weaknesses, it can be difficult to decide on a particular method for a particular application. Moreover, in most published work the parameters underlying the algorithms are typically hand-tuned until satisfactory performance is achieved, and the methods are presented without a clear picture of what the important differences, advantages, drawbacks, and trade-offs are.

This work aims to more systematically approach the topic, discussing each component of the tracking system and how it affects performance. We hope that this effort will help develop a more unified view of RT-FD-SPT, making it more accessible to a wide range of scientists. The review also aims to be more complete than the previous three reviews, since especially techniques focusing on improved feedback control and position estimation have been overlooked thus far. Image-based SPT has been comprehensively reviewed, but the same cannot be said for RT-FD-SPT. We address control theory and parameter estimation, two aspects of RT-FD-SPT that are generally only cursorily considered by most research in the field, and look at the criteria used to evaluate performance when choosing a method. Lastly, we aim to highlight the potential of concurrent spectroscopy, which has also been under-emphasized in the research to date.

The review is structured as follows: Section 2 presents a survey of RT-FD-SPT techniques, while Section 3 discusses applications in microscopy and spectroscopy. Sections 4 and 5 respectively discuss the aspects of control and estimation theory relevant to RT-FD-SPT, while Section 6 discusses system characterization and calibration. Finally, Section 7 discusses the performance of the different methods and Section 8 presents an outlook to the future of the field.

2. Survey

RT-FD-SPT differs from conventional image-based SPT in that the particle in question is tracked in real time, with a feedback loop keeping the particle in the observation volume. This offers several advantages. Firstly, particles can be tracked over long distances in all three dimensions, while maintaining excellent spatiotemporal resolution (the temporal resolution is routinely ~ 1 ms or less). Secondly, the particle can explore a variety of environments, whereas conventional SPT is typically limited to a single region of interest. This is due in part to the long range capability (typically ~ 10 μm, with a range from 50 μm to a few centimeters [17]) of RT-FD-SPT and to the fact that the tracking parameters can be changed in real time to adapt to different environmental circumstances. Lastly, keeping the particle in a small observation volume allows concurrent single-molecule spectroscopic measurements to be made. The main drawback of RT-FD-SPT over conventional SPT is that only one particle can be tracked at a time, except when diffusion is slow enough that multiplexing is possible. There is also the possibility of losing the particle prematurely. Furthermore, RT-FD-SPT does not, by itself, provide contextual imaging. This can be provided by concurrent techniques (see Section 3), but this adds cost and complexity. Of course, such concurrent imaging may well allow conventional SPT, and in this way, the two techniques are complementary and can even be used in the same experiment. Figure 1 shows a taxonomy of current RT-FD-SPT techniques, which can be grouped into single-detector and multi-detector methods.

Figure 1: — A taxonomy of current 3D RT-FD-SPT techniques. The class of SPT microscopes that use feedback control are depicted on the far left. These microscopes use information gained from photon detections to control the optics and lasers enabling it to lock onto a moving particle or molecule. This class can be grouped by the number of detectors used. Methods that use a single photodetector can be further grouped based on how the laser is scanned. Three groupings of scanning methods emerge: deterministic scan, constellation scanning pattern, and reactive scan. Deterministic scanning moves the laser along a predetermined path; depicted is the 3D-orbital method. The results are then used to adjust the location of the center of the scan using feedback. Constellation methods dwell the laser at a set of predetermined locations; depicted is a typical constellation. The results are used to adjust the location of the center of the pattern using feedback. Reactive scans determine the next direction of the laser through feedback determined by the most recent photon detections; depicted is the extremum seeking method. The group of methods using multiple photodetectors can be further broken down into methods that use a small number of detectors, and those that use cameras. Parallel detection methods utilize multiple photodetectors to determine the x, y, and z positions simultaneously; depicted is the optical cantilever technique, showing the detection pattern for determining x and y positions on the left and the channel for determining the z position on the right. Local imaging methods use a reduced region of interest on a scientific imaging camera; depicted are the photon counts of a row of pixels of the bi-plane method.

2.1. Single-detector methods

With a single detector, the position is determined by using time-varying excitation. The beam-scanning family of techniques (Figure 2a) are based on fast scanning of the excitation beam, using piezoelectric or galvanometer mirrors, or with electro-optical or acousto-optical deflectors (EODs/AODs), with the latter two options being faster. Axial scanning can be performed by moving the objective [18], by using an electrically tunable lens (ETL) [19], or by using a tunable acoustic index gradient (TAG) lens [20], with the speed increasing in the same order. Another way to obtain axial information is to switch between two focal planes using two beams that are switched on and off exactly out-of-phase using acousto-optic modulators (AOMs) [21]. It is also possible to scan the sample instead of the beam using a two- or three-dimensional piezo stage [22, 23]. This is conceptually equivalent, but has some practical differences. Scanning the sample with a stage is typically slower than scanning with a fast mirror or EOD/AOD. There is also the common concern that fluid dynamics in the sample due to the movement of the entire sample interferes with the particle motion. While this has been shown not to be a problem in practice [24], beam scanning remains more popular for fast scanning.

The orbital scanning method was initially proposed by Enderlein [25], and first implemented independently by the groups of Gratton [18] and Mabuchi [26]. It was also used in one of the early versions of the Anti-Brownian ELectrokinetic (ABEL) trap [27] (see Section 2.3 for more about the ABEL trap). This method involves scanning the beam in a circle around the particle and measuring the fluorescence signal with a point detector such as a single-photon avalanche diode (SPAD). The position of the particle can be inferred from the resulting sinusoidal temporal variation in the intensity signal. An alternative to orbital scanning is “local raster scanning”. This was first used for Levi and coworkers’ “Scanning fluorescence correlation spectroscopy (Scanning-FCS)” [28]. Wang and Moerner [29, 30] implemented this method using a knight’s tour pattern in the ABEL trap and this approach was extended to 3D by Hou et al. [20] to create 3D Dynamic Photon Localization Tracking (3D-DyPLoT). The advantage of this method is that it enables a larger tracking area, allowing tracking of faster particles. However, this comes with a trade-off of worse precision [31]. Another disadvantage of the Knight’s Tour method is that it involves discontinuous scanning. Photons detected between scan points are not used for position estimation, and thus the scanning hardware needs to be very fast in order to spend as little time as possible moving between points. This adds significantly to the cost of the hardware.

An alternative to a fine-grained scan like the orbital or local raster scan methods is to scan the beam through a constellation of a small number of points, as first implemented by Shen and Andersson [32, 33, 34], using a six-point constellation in 3D [35]. This also requires fast scanning hardware, but has the advantage that the measurement locations can be chosen so as to maximize the position information gained from the observed photons [36, 37]. The same approach is taken with the superresolution technique MINFLUX [38, 39], and a “hybrid” approach was presented by Zhang and Welsher where a small number of XY points were used while scanning continuously in the Z direction [37].

An alternative to scanning the laser beam or sample stage is to use pulsed interleaved excitation (PIE) (Figure 2b). It was originally developed for techniques such as FCS and Förster resonance energy transfer (FRET) [40]. The use of PIE for RT-FD-SPT was first suggested by Dertinger et al. [41], and experimentally demonstrated by Davis et al. [42, 43], by using four alternately pulsed laser diodes with foci placed in a tetrahedral shape around the particle. The arrival times of the detected photons are then used to determine the position of the particle. Perillo et al. [44] implemented the idea in their microscope - named “tracking single particles using nonlinear and multiplexed illumination (TSUNAMI)” by using a femtosecond laser beam that is split in four and sent along physical delay lines before being recombined. For time-gating the photon arrivals, a time-correlated single photon counting (TCSPC) system was used, as the pulses in different channels were only 3.3 ns apart. The technique was recently expanded to incorporate two-color detection [45]. A pulsed interleaved version of MINFLUX was also recently presented [46], and PIE has been used for a three-dimensional version of the ABEL trap [47].

Instead of scanning along a deterministic pattern or using PIE, it is also possible to use a control system that generates the scan path dynamically. This reactive scan method was first implemented by the Andersson group in the form of an extremum seeking controller [48, 49] that maximizes the detected intensity. Recently, Vickers and Andersson presented a controller design that aims to maximize the positional information gained from the measurement [50].

2.2. Multi-detector methods

Detection path-based methods can be based either on parallel detection or on local imaging. Parallel detection tracking was first developed by the groups of Yang [24, 51] and Werner [52, 53, 54, 55]. The approach taken by the two groups differs somewhat. The Yang group initially used a quadrant detector [24], but later developed the so-called optical cantilever [51] (Figure 2d). The latter method made use of two detectors each for X and Y, as well as a detector for Z. The detected signal is split 50/50 between the XY and Z paths. The XY signal is split 50/50 again between X and Y and projected onto the centers of two prism mirrors with two detectors each. When the particle is centered on an axis, the signal is equal in both detectors for that axis. When the particle moves, the relative increase of the signal in one detector on an axis indicates the direction of motion. For axial position information, the signal is focused through a slightly offset pinhole onto the Z-detector. This gives a linear relationship between the detected intensity and the Z-position. The Werner group’s version uses two pairs of detectors coupled to optical fibers, with the other ends of the fibers placed at two different focal planes, so as to correspond to a tetrahedron in the sample volume (Figure 2c) [52, 53, 54, 55].

Local imaging refers to the use of a camera, such as a CCD or scientific CMOS camera, to image the particle and determine its position. The bi-plane method was developed by Juette and Bewersdorf [56]. They used two image planes in order to achieve 3D-localization, using different regions of an EMCCD camera to image both planes at the same time (Figure 2e). By using only a 5 × 5 pixel region for each image, the readout and analysis of the image were computationally fast. Another imaging method that made use of real-time feedback was developed by Spille et al. [57]. This method combined light sheet illumination with astigmatic detection to determine the axial position. A Z-axis piezo stage was used to refocus the sample in real time.

2.3. Complementary methods

There are many methods that share some features with RT-FD-SPT. One is wide-field image-based tracking, which is a well established and widely used technique. Another is the ABEL trap, which is sometimes (erroneously in our opinion) considered to be an SPT technique. It makes use of exactly the same type of position measurement and feedback, except that electric forces are used to reposition a particle in solution, instead of moving the sample or illumination beam. This allows for a much faster response time (particles with a diffusion coefficient D ≈ 100 μm² s⁻¹ have been trapped [30]). However, the particle needs to be isolated in solution, and this technique can thus not be used in vivo. It can, however, be used to measure diffusion and electrokinetic mobility [30], and for performing spectroscopy on particles in solution [58]. There are many other related trapping techniques, based on physical, optical or thermodynamic principles [59]. There are also techniques that use confocal detection to determine particle trajectories, but without using feedback. These include a multi-detector method similar to multi-detector RT-FD-SPT [60, 61], and a method that uses a single detector paired with a statistical method to extract the most likely trajectory from the photon counts [62].

3. Applications

Far more numerous than the approaches to RT-FD-SPT are the possible applications. RT-FD-SPT has been applied in many different biological contexts and its utility is mainly twofold. Firstly, it allows the observation of particle motion in three dimensions, at a high temporal rate and with an excellent signal-to-background ratio. To this end, the experimental setup is built around a microscope and is comparable to setups used for conventional SPT. This is usually combined with other, complementary imaging modalities. Secondly, spectroscopic measurements can be made on tracked particles. Both of these factors have proven to be very useful and we discuss them separately here.

3.1. Microscopy and direct tracking-based applications

In one of the first examples of feedback-based SPT, So et al. [63] used two-photon excitation to study phagocytosis, by tracking 2 μm latex spheres as they were captured by macrophages. Peters et al. [64] used a setup based on the deflection of a focused laser beam by a polystyrene sphere to measure motion of LFA-1 molecules in the cell membrane. Soon after the first demonstrations of the orbital tracking method, it was used to study phagocytosis of fluorescent beads by fibroblasts [18]. It has since been used to study properties such as cell mechanics [65, 17], membrane dynamics [66], transport [67, 68, 69, 23, 70, 19], and DNA transcription [71, 72]. This is often done alongside imaging techniques such as light microscopy, fluorescence microscopy, two-photon microscopy, or superresolution microscopy. The method has also been used to track nanoscaled structures such as changes in the shape of microvilli [73], the dynamics of endolysosomes along microtubules [70], and for tracking entire mitochondria in zebrafish larvae [17, 74]. The Knight’s Tour scanning pattern, originally implemented by Wang and Moerner [29] and extended to 3D by the Welsher group, was used to track virus-like particles landing on the cell membrane [75] and cellular trafficking of mRNA-encapsulating lipid nanoparticles [76]. MINFLUX has been used to study the diffusion of a ribosomal subunit in E. coli [38] and to image the Nup96 protein in the nuclear pore complex [39, 77], the post synaptic protein in neurons, and the MICOS complex in the mitochondrial inner membrane [78]. Tetrahedral excitation tracking has been used to study endocytosis and subcellular trafficking [44, 79] as well as the landing of nanoparticles on the plasma membrane [45]. This technique was also combined with two-photon laser scanning microscopy (2P-LSM) in a very similar manner as for 3D multi-resolution microscopy (3D-MM). Multidetector tracking has been used for tracking membrane receptors [80], including with simultaneous spinning-disk imaging [81]. 3D-MM combined multi-detector tracking with 2P-LSM to achieve a detailed view of the cellular uptake process [82].

Figure 3 shows two examples of RT-FD-SPT combined with concurrent imaging. In Figure 3a an orbital tracking trajectory of an artificial virus in a live cell is shown with concurrent widefield imaging at two different image planes [67]. In this experiment, RT-FD-SPT allowed high-precision tracking of the virus over a 100 μm range, while the concurrent imaging was used to distinguish between motion of the virus along a microtubule and movement of the microtubule itself. Figure 3b shows a multi-detector tracking trajectory of a peptide-coated nanoparticle as it landed on an NIH-3T3 cell, with the cells imaged using 2P-LSM. Here, RT-FD-SPT provided precision tracking at μs time resolution of a fast moving (2–5 μm²s⁻¹) particle, while the 2P-LSM provided 3D visualization of the cell surface.

The main benefits of RT-FD-SPT over traditional SPT in all of these applications is a very long tracking range and good 3D tracking ability, with excellent spatiotemporal resolution. Some of the studies also exploited the ability to concurrently measure fluorescence spectra [66] or intensities [71]. The former proved useful as a way to distinguish the label of interest from a different fluorescent object. The latter was used in various ways to investigate the dynamics and interactions of the tracked particles. The advantage of the combination of RT-FD-SPT with other imaging modalities is illustrated by Welsher and Yang [82] as a “folding” of spatiotemporal regimes: RT-FD-SPT is able to capture dynamics at nanometer length scales and microsecond time scales, while concurrent imaging captures dynamics on micrometer length scales and second time scales.

3.2. Spectroscopy-based applications

SMS has proven to be a powerful tool for studying biological systems, allowing the detection of unsynchronized dynamics of molecular-scale systems that are averaged out in bulk measurements. It has been applied to the study of a wide variety of biological systems, including DNA [83, 84, 85], enzymes [86, 87], molecular motors [88, 89], and photosynthetic proteins [90, 91, 92]. With SMS, it is possible to observe, for example, conformational changes in proteins [93], transcription in single DNA molecules [94, 95], enzyme reactions [96, 87], and switches between various functional states in light harvesting complexes [97, 98, 99]. SMS is commonly used as an umbrella term for describing any type of single-particle detection (except imaging) using optical radiation, mostly fluorescence [13]. Measurable time-dependent information-bearing parameters include the fluorescence brightness, spectrum, lifetime, polarization state, and photon correlations. First-order photon correlations enable different variants of FCS, while second-order correlations enable measurements of photon bunching and antibunching. Two-color fluorescence measurements allow the investigation of FRET. More advanced spectroscopic techniques such as excitation-emission spectroscopy [100] and ultrafast spectroscopy [101, 102, 103, 104, 105, 106] have also been performed at the single-molecule level, as has Raman spectroscopy [107].

Some SMS methods have already been combined with RT-FD-SPT. Mabuchi’s group introduced “tracking-FCS” [26] and used it to investigate DNA conformational fluctuations [108] . The same group also measured photon antibunching on diffusing quantum dots (QDs) [21]. Werner’s group performed similar measurements on QDs [109, 81] as well as on fluorescent proteins [110]. Yeh’s group showed antibunching of dye-labeled DNA in solution [111] and in live cells [112] and Welsher’s group showed antibunching of diffusing single dye molecules [95]. The latter group also used lock-in filtration of fluorescence brightness measurements to investigate the nanoparticle protein corona [113]. Werner and coworkers demonstrated the first direct measurements of fluorescence lifetimes with RT-FD-SPT [109], and this capability was also shown by the group of Yeh [44], who applied it to the measurement of DNA hybridization kinetics in solution [111] as well as in live cells [112]. Similar measurements have also been performed on DNA origami structures using p-MINFLUX [46]. Recently, Yang’s group used lifetime information to improve their system’s signal-to-background ratio by separating long-lifetime fluorescence from the short-lifetime background signal [114]. Keller et al. [115] used a total of eight detectors to perform single-molecule FRET (smFRET) measurements with simultaneous tracking on dye-labelled DNA. To our knowledge, the only measurements of emission spectra using RT-FD-SPT were done by Hellriegel and Gratton [66], who measured two-photon emission spectra of GFP-labeled uPAR in live cells .

Incorporating spectroscopic measurements into RT-FD-SPT typically involves additional detector hardware or an additional beam. To measure lifetimes, a TCSPC system synchronized with a fast pulsed laser needs to be added. The pulsed laser can also be used for tracking. For antibunching measurements, an additional point detector is needed together with correlation hardware, typically a TCSPC module. FRET measurements also require an additional detector. To measure spectra, a spectrometer is needed, which consists of a dispersion device (a diffraction grating or prism) and a line detector or 2D sensor such as a sensitive scientific CMOS or CCD camera.

Yang and coworkers introduced the idea of a separate beam for spectroscopy [24], a technique that was subsequently also used by McHale and Mabuchi [108, 116]. One advantage of this approach is that the tracking beam can have a different wavelength such as NIR, which can be used at a higher intensity without causing photodamage to the sample [24]. This could involve scattering or interferometric scattering (iSCAT)-based detection (see Section 8.1) or a separate tracking label to ensure that tracking and spectroscopy have separate photon budgets. For example, if one is interested in measuring spectra, all the photons in the spectroscopy channel could be sent to a spectrometer. For naturally fluorescent samples, a bright fluorescent label or the scattering signal could be used for tracking, while the comparatively weak natural fluorescence is investigated using spectroscopy. Another interesting use of two beams was the case of tracking-FCS measurements on DNA, where the tracking beam was larger than the molecule, whereas the probe beam was smaller than the molecule, making it sensitive to intramolecular motion, the study of which was the purpose of the experiments [108].

Figure 4 summarizes spectroscopic applications of RT-FD-SPT and shows a few illustrative examples of spectroscopic measurements that have been made with RT-FD-SPT. Figure 4a also indicates the additional hardware that is needed to add the technique to an RT-FD-SPT setup.

Although the results so far are impressive, they have only scratched the surface of the wealth of information that could be obtained when combining RT-FD-SPT with SMS. To give just one, illustrative example of the impact that SMS studies can have on our understanding of the properties and functions of proteins and complexes at the nanometer scale, consider light-harvesting complexes. Owing to their autofluorescence, no fluorescent label is needed, enabling the use of fluorescence as a direct probe of the properties and protein conformational dynamics of the complexes. The light-harvesting complexes of a wide variety of photosynthetic organisms have been investigated, including bacteria [118, 119, 120]— especially cyanobacteria [121, 122, 99, 123]—, plants [124, 125, 98], and algae [126]. Multiple spectroscopic parameters are typically used, often simultaneously. For example, correlations between changes in fluorescence brightness, lifetime, and spectra have been used to identify and characterize new physiological states [124, 125, 98, 99, 126, 127]. It has also been demonstrated how the interactions between different individual proteins can be controlled [128], how metallic nanoparticles can drastically tune the photophysical parameters of these complexes [129, 130], and how ultrafast spectroscopy can resolve rapid fluctuations and quantum-coherent processes in these systems at the single-molecule level [102, 103]. RT-FD-SPT could be used to perform measurements such as these in a close to in vivo environment or even in live cells. There would be drawbacks, the main ones being a decreased SBR and shorter measurement times due to tracked particles being lost. However, the advantage of the results being much more physiologically relevant would most likely outweigh these drawbacks for most experiments.

4. Feedback Tracking and Control

The core of RT-FD-SPT is the tracking feedback control system. Feedback control enables the optical system to lock onto and track the target particle, extending the duration over which one can take meaningful optical and spectroscopic measurements. The design and implementation of the tracking controller as well as the tuning of its parameters can have a profound impact on the overall performance of the tracking system. In this section we discuss control strategies for RT-FD-SPT and discuss the challenges and limitations that must be considered when designing a controller for RT-FD-SPT.

4.1. Controller Architecture

There are two main architectures for control in RT-FD-SPT, one based on reference point tracking (Figures 5a-5b) and one based on a signal maximization approach (Figure 5c). Both rely on feedback to steer the tracking system and thus inherit the benefits that feedback brings, including robustness to noise, to external disturbances, and to changes in the system dynamics. In both of these architectures, at the innermost layer is the actuator hardware (which could be, for example, a piezoelectric stage, EOD, AOD, or a TAG lens) to realize the fastest time scale motion. These actuators typically come with manufacturer-designed controllers and for the purposes of this discussion are assumed to be a black box with an output that faithfully follows its input.

Figure 5: — System block diagrams for typical reference point RT-FD-SPT. Both the (a) single and (b) multiple detector systems feedback an estimate of the particle location while the (c) maximization-based approaches actuate based purely on the measured signal.

In the reference point tracking approach, the reference point is given by the estimated position of the particle (see Section 5 for details on generating such estimates from the optical measurements). This scheme has been used in multi-detector tracking methods, where the actuated variable is typically the center of the detector array, and in single detector techniques, where the actuated variable is the center of a scan pattern. The goal of the controller is to drive the actuated variable to the reference point and the design of this controller is (at least conceptually) independent of whether the system is moving the sample towards a fixed excitation/detection setup or moving the excitation/detection setup towards the sample.

A common controller choice in the regulation approach is a Proportional-Integral-Derivative (PID) controller. PID controllers have been implemented with good results in local imaging tracking [56], local scanning tracking [63, 65, 20, 75, 95], multi-detector tracking [24, 51], measurement constellations [38, 44, 131, 45], and orbital scanning tracking [132, 18]. One of the major reasons for the success of PID is that there is a simple interpretation for each of the three components of the controller, allowing intuitive adjustment of the parameters to achieve good performance. The P term applies a feedback signal that is directly proportional to the measured error and, in general, increasing this gain leads to a faster response with lower final error. However, making it too large can drive the system to “overshoot” the goal position and create a strong oscillatory response with a long transient. The I gain serves to reduce the final error of a system but can lead to instability (where the system does not track the particle) if tuned too high. The D term provides an anticipatory response, reacting as soon as the error begins to change, with a magnitude that is proportional to how fast it is changing. However, derivative action tends to amplify any noise in the signal and can lead to undesirably sharp responses that can excite nonlinearities in the actuators. Because of these downsides, in practice many applications use either just P control [28, 132, 18], PI control [38, 44, 131, 45, 95], or even just I control [20]. A deep and thorough discussion of PID control can be found in [133].

Achieving good tracking performance on a specific system depends on good tuning of the controller parameters. Many of the published algorithms in RT-FD-SPT involve hand-tuned parameters, which are adjusted until satisfactory performance is achieved based on either physical experiments or simulations [131]. However, this approach can be quite time-consuming and the level of performance achieved depends strongly on the skill of the person doing the tuning. For PID control, an alternative is to apply one of a variety of heuristic methods that use easy-to-measure experimental values (such as a rise time or the period of an induced oscillation) to calculate gain values that provide a good balance of speed and stability for the particular system. Many such rules have been developed (see, e.g., [134]) with perhaps the best known being that of Ziegler-Nichols (see, e.g., [135]). While some amount of hand-tuning is typically still needed after applying a heuristic, applying these rules ensures that this tuning starts from values that already provide reasonable performance.

While PID often works well in practice, it is a fairly conservative controller, providing stability and good robustness against noise, disturbances, and uncertainty at the cost of speed of response. There are several alternatives based on model-based control that can outperform PID but which, as the name implies, require an accurate model to ensure good performance and run the risk of causing instability if those models are not reasonably close to the true system. Developing controllers based on these techniques often requires a strong background in control engineering and thus these schemes are outside the scope of this review. One exception worth noting is that of the Linear Quadratic Regulator (LQR). LQR develops a controller designed to minimize a cost function given by a weighted combination of the tracking error (seeking to drive it to zero) and the magnitude of the control signal (seeking to keep it small). While more challenging to use than simple PID, LQR has the benefit of a built-in notion of optimality and the ability to easily modify the controller to ensure the control signal does not hit physical constraints such as voltage bounds or actuator slew rate limits (though without providing a theoretical guarantee on satisfying any such constraints). It has been applied to good effect in several RT-FD-SPT algorithms [136, 32, 33, 34, 35].

The second class of controllers, based on signal maximization, are inherently nonlinear but come with the benefit of not relying on any estimate of the particle position. Rather, they work directly with the measured signal, steering the measurement volume either to maximize that signal or to converge to a periodic pattern with a constant signal level. An early version of this idea was used to automatically find and position a laser for spectroscopic studies of single molecules [137]. The controller in this case used a simplex method to measure the photon count rate, and used a gradient ascent to align the laser to the molecule automatically. More recently, an extremum-seeking controller was developed to converge to an orbit around a fixed fluorescent particle [138, 48, 139, 140, 49, 141, 142]. Through simulations and experiments, it was shown that this controller naturally switches between two distinct behaviors, converging to the desired orbital motion for slow moving particles and transitioning to a chasing behavior as the speed of the particle increases. While the ability to operate directly on the intensity signal, thereby separating particle localization from control, is appealing, the nonlinear nature of these approaches makes theoretical analysis and parameter tuning more difficult than PID. Still, the benefits of these nonlinear approaches can be valuable. For further guidance on the convergence properties of the extremum-seeking approach and on selecting controller gains, see [141, 143]. Ultimately, the decision to select a signal maximization method, with its added complexity to tune, due to its nonlinear nature, but ease of implementation, due to the lack of an online estimator, depends on a variety of factors. Unfortunately, there is no existing detailed, experimental comparison between these methods and those based on reference point tracking; this represents an opportunity for future research.

One often overlooked aspect common to all control architectures is the initialization step, namely how to select a particle for tracking. There are two primary methods for doing this: image-based and threshold-based. Under the image-based approach, an image of the sample is captured and displayed to the user. That user then selects a particle of interest to begin the tracking process [33]. This allows the user to ensure “interesting” particles are selected but can be severely constraining in terms of particle speed, since the particle cannot move significantly in the time it takes to perform the capture, display, and selection steps. In practice, this method works for particles with diffusion coefficients of about 0.1 μm² s⁻¹ or less (see Supporting Information for more details). Under the threshold-based approach, the user selects a location but not any individual particle. The detection volume is moved to that location and monitors the photons counts. When a sudden increase over a given threshold is detected, the tracking controller is enabled [38]. Since selection is no longer constrained by user reaction time, faster particles can be tracked. In addition, once a particle is lost, the system can switch back to dwell mode and wait for the next threshold crossing, automatically picking up another particle. However, in this approach, background noise and detector counts force a lower bound on the threshold to avoid too many false detections, possibly making the system somewhat insensitive to dim particles. While one can apply a systematic approach such as constructing a likelihood ratio test to determine a threshold that balances sensitivity against false detections, in practice the value tends to be chosen through trial and error. In addition, many fluorophores exhibit blinking or temporary dark states. Enabling the system to wait for a short time for the particle to enter a bright state can extend the duration of tracking a particle.

4.2. Control Challenges and Limitations

The performance of RT-FD-SPT algorithms is ultimately limited by several physical challenges and trade-offs that must be made when designing the control system. To help understand these trade-offs, we organize this discussion around a time budget as shown in Figure 6. Within each update step of the controller there are three steps that must occur: detection and collection of the photon signal, implementation of the algorithm to determine the actuator command, and then, finally, the actual motion of the physical hardware in response to that command. It is also important from a practical implementation point of view to include a margin in the update time where no work is done; this allows the system to absorb small jitters in the timing and allow the computational hardware time to perform other functions (such as storing data or communicating to other systems in the experimental setup). The update rate must therefore be slow enough to allow enough time to perform all three steps, as well as to acquire any spectroscopic data desired, while also being fast enough to track the particles of interest over a long enough duration to acquire sufficient measurements to probe the questions of interest. Note that most, if not all, RT-FD-SPT approaches make the assumption that the particle is quasi-static, that is, that the time scale of the algorithm (as defined by this update rate) is much faster than the motion of the particle.

Figure 6: — Temporal budget. Inside each update step of duration Δt, any RT-FD-SPT algorithm must collect photons from the tracked particle, translate these photons to a desired update of the tracker, and then implement this update through actuation. Under stable tracking, the algorithm will keep the particle inside the detection volume of the algorithm (green trajectory); if the particle is too fast relative to the update rate, or the estimation error too large, then the particle may move out of the detection volume before an update is completed, leading to loss of tracking (blue trajectory).

Satisfying this quasi-static assumption is important to achieving the expected performance of RT-FD-SPT. An upper bound on the maximum speed of particle dynamics that can be reliably tracked can be developed by assuming that the particle must stay within the detection volume within a single time step. If the particle moves out of this volume prior to the next measurement, then that measurement will yield no information about the particle and it is lost. The maximum distance traveled within one time step is described by a probability distribution defined by the particular dynamics of the particle. Assuming the particle starts at the center of the detection volume at each step (perfect tracking), the notion of the first passage time to reach the edge of the detection volume sets an upper bound on the particle speeds that can be tracked [22, 141]. This upper bound can be increased either by increasing the update rate or by increasing the detection volume. The update rate is limited by the time required for the three steps of RT-FD-SPT; the limitations for each of these steps are described further below, with typical values for completing all three steps ranging from 125 μs [38] to 5 ms [117, 44]. Increasing the effective detection volume can be done by expanding the size of the scan or constellation used in the different approaches, with typical length values ranging from about 400 nm [38] to about 3 μm [20]. For example, the work in [21] expanded the size of their orbital scan to enable antibunching measurements on freely diffusing fluorophores with diffusion coefficients of 20 μm² s⁻¹; this came at the cost, however, of a loss in localization performance with an accuracy on the order of 200–300 nm. This trade-off exists on a continuum and it can be shown (see Supporting Information) that the loss in localization performance is proportional to the square of the effective size of the detection volume when keeping detected photon counts constant and to the fourth power of that size when keeping the illumination power constant. It is also important to note that collecting photons over a larger effective detection volume implies a larger distance to be covered for scanning approaches, necessitating either very fast scanners (as in, e.g., 3D-DyPLoT and its variants [20, 75, 95]) or a larger portion of the time budget allocated to photon collection. Of course, meeting a desired speed of tracking is only one element in selecting the update rate and how to spend the time budget it defines. It is important to understand the trade-offs involved in giving more or less of that time budget to each of the steps in the update.

Photon collection:

The first step in an update is the collection of photons. Enough informative photons must be collected for either the online localization algorithm to provide accurate estimates of the particle location or for the maximization-based controllers to accurately estimate the direction of signal increase. Because photon detection is fundamentally a Poisson-distributed counting process [144, 145, 146, 147], the acquired signal is always noisy with a Signal-to-Noise Ratio (SNR) that improves as $\sqrt{N}$ , where N is the number of photons detected. The SNR can be improved by increasing the (average) number of photons acquired, either by increasing the photon emission rate or by increasing the time spent collecting photons. The first approach is limited largely by concerns of photobleaching and phototoxicity and is often not a viable approach when using many fluorescent dyes or autofluorescent proteins. The second approach comes at the cost of reducing the time available to the other steps in the update or in decreasing the update rate and slowing down the overall controller. The effectiveness of this is also reduced by the reality that the particle is in constant motion during the photon acquisition process, leading to motion blur in the data that degrades overall performance. While some estimation algorithms include at least partial efforts to overcome motion blur [148, 149], it remains a challenge in RT-FD-SPT and is one of the drivers for making the quasi-static assumption.

In most RT-FD-SPT approaches, measurements from multiple locations are needed at each time step, further dividing the time budget, as all of this data must be collected inside a single update. Most implementations, then, place at least some emphasis on how fast data can be acquired from different locations, some by using multiplexed detection [24, 51, 14, 150, 56, 151] or pulsed interleaving [44, 111, 152], others by employing high-speed AOD [21] or EOD scanners [38, 20], and still others by using advanced control methods to achieve fast motion [35, 48]. Note that when using these methods, multiple cycles of photon collection can and does occur during the time budget. This provides a high temporal resolution (also known as time to localization) down to 20 μs. Unfortunately, this temporal resolution is often conflated with the update rate of the tracking controller, which often occurs at a slower rate (~ 5 ms) due to the smaller bandwidths of the actuators used for tracking [24, 44, 20, 38].

More recently, some approaches recognize that not all photons are equally informative with respect to localizing the particle. Using ideas from optimal estimation theory, particularly the Fisher Information Matrix and the corresponding Cramér-Rao lower bound, one can ensure the photons that are collected are useful with respect to the localization task [145] (see Section 5). This idea has been shown to improve precision by approximately 20% in 3D localization relative to uniform sampling [37] and used to design control algorithms for steering the excitation source [50].

Finally, one must also consider how photon collection affects other aspects of estimation and system performance. The detection pattern used to localize the molecule must provide data that enables the localization algorithm to produce a unique result. In most methods, a symmetric pattern provides this. However, the shape of the beam used in MINFLUX requires an additional measurement at the center of their circularly-symmetric pattern to ensure a unique solution [38]. Another aspect to consider is the localization uncertainty compared to the characteristic width of the detection volume. In essence, the total uncertainty from all sources, including localization error, motion blur, and particle dynamics, must be smaller than the width of the detection volume to ensure tracking is maintained. Additionally, while the focus is generally on tracking, the information obtained is often used to estimate other parameters. While image-based approaches to SPT have addressed the effect of localization uncertainty on such estimation through both Fisher information [153] and Monte Carlo simulations [154], this remains an open question in RT-FD-SPT.

Algorithm:

The second step in an update is to perform all the calculations necessary to translate the measurements into an action. For most schemes, this includes filtering to reduce the effect of noise in the signal, online estimation of the particle position, and then calculation of the control signal. Note that some level of filtering is important since, as noted above, the photon signal is inherently noisy. The fluctuations in these measurements persist through the estimation and thus are injected into the feedback loop of the controller. It was shown experimentally in Ref. [75] that a tracking controller that is more sensitive to changes in the reference point shows an increased localization uncertainty for a static particle. When the controller is made less sensitive, the static particle localization precision improves, but at the cost of speed of response. Filtering of the signal using, e.g., a Kalman filter [136, 22, 32, 20], helps to mitigate the noise, potentially allowing for higher gain controllers, but needs additional computation in the update step. Note that the maximization-based approaches avoid the need for online estimation but may still benefit from filtering. In order to keep the computation time as short as possible, many techniques use linearized estimators [136, 38] or simple difference-over-sum estimators [25, 109, 44] to reduce the computational load, combined with fast hardware such as lock-in amplifiers [26] or field-programmable gate arrays (FPGAs) [38, 20] to do the calculations as fast as possible.

Actuation:

The final step in an update is to apply the control signal to the actuation hardware and for the system to complete the desired motion for the current update. The time scale for this step is defined by the bandwidth of the actuation system. In many cases, particularly with piezoelectric components and their manufacturer provided controllers (which generally value robustness and safety over speed), this is the bottleneck limiting the overall speed [15]. Significant improvements in bandwidth can be achieved by designing custom low-level controllers but doing so requires both access to the direct inputs to the actuators and experience in control systems design. In general, then, selecting fast hardware and making the most out of that hardware through appropriate tuning of the low-level controllers and of the algorithm parameters helps to minimize the impact of this step on the overall time budget. One major challenge here, however, is that the gains that give the best tracking performance are a function of the experimental conditions, including considerations such as the speed of the particle, the photon detection rate, and the background count rate. For example, it was shown through simulations that the tracking error of TSUNAMI depends on both the controller gains and the diffusion coefficient of the tracked particle [131]. The best gains are a function of diffusion coefficient; gains that were too large amplified the estimator noise while gains that were too small yielded insufficient sensitivity to changes in the particle location. Even more challenging, the experimental conditions typically change during the course of experiment. Two common examples of this are photobleaching, causing a reduction in the detected photon rate over time, and changes in the diffusion coefficient due to binding events or changes in molecular crowding [82]. In these cases, the best tuned controller can become ill-suited to the changing experimental conditions.

To overcome these challenges, controller parameters can be adapted in real time in response to changing conditions. In some approaches, the size and shape of the scan pattern is modified based on the current uncertainty about the particle location; in essence, widening the scan pattern improves detection while tightening the scan pattern improves localization precision [38, 39]. Note that for beam scanning methods, adapting the scan pattern also requires adapting the beam diameter to maintain good localization [145, 50]. Other methods have adjusted the scanning volume based on the (estimated) diffusion coefficient of the particle, increasing localization precision when that coefficient is small and increasing the probability of detection when it is large [75]. The decision to adapt based on localization uncertainty or based on diffusion coefficient depends largely on which source of error dominates a particular application. While the use of such adaptive controllers is still limited, they show promise for improving the overall performance by adjusting to experimental realities.

5. Estimation and Fisher Information

5.1. Estimators

Estimation is a key component in RT-FD-SPT. It plays two related but distinct roles: to provide the position of the particle in reference point tracking controllers (see Section 4), and to provide information to fulfill the scientific aim. The specific scientific aim is, of course, application dependent and includes estimating the position of the particle and its diffusion coefficient [155, 49, 75], detecting when the diffusion coefficient changes [82, 156], determining dynamics of conformational changes [157], and estimating changes in photophysical states [158].

Algorithms intended for direct use in the feedback controller use online estimation and are thus restricted to causal techniques as they can only use measurements up to the current time. It is also vital that the estimator complete its computations quickly, both to support a fast closed-loop rate in the tracker (see Section 4) and because estimation algorithms almost invariably assume the particle is static during the measurement process. Many scanning-based or multidetector-based approaches also assume that the tracker keeps the particle close to the center of the pattern, allowing for a linearization of the observation model around this point so as to develop efficient estimators. Because speed is typically one of the primary considerations in the controller, it is often acceptable to sacrifice some accuracy for speed, relying on the robustness of the controller to handle the resulting error. At one end of the trade-off between accuracy and speed are simple centroid-style algorithms [63, 65, 150, 56, 151] and difference-over-sum estimators [24, 51, 14, 109, 110, 115, 82, 82, 15]. Slightly more complex are algorithms that take some advantage of knowledge about the shape of the point spread function (PSF) of the instrument, including the fluoroBancroft algorithm [32, 33, 34, 35, 36] and assumed density filters [159, 20, 75, 95]. At the other end of this trade-off are methods based on optimal estimation, including maximum likelihood estimation (MLE) [22, 38, 50], minimum mean squared error estimation (MMSE) [38], and least squared error estimation (LSE) [63, 65, 160, 32, 33, 34, 35, 36, 161]. In many cases, the estimation algorithm is coupled with a filtering scheme (often a Kalman filter) to help remove noise from the estimates [22, 32, 33, 34, 35, 36, 162, 20, 75, 16, 95].

While the estimation error has a clear impact on the quality of tracking, its effect on the feedback controller in a RT-FD-SPT system has not been explicitly considered in the literature. As noted in the previous section, a simple upper bound on the allowable error can be defined by ensuring that it is smaller than the radius of the detection volume of the system since otherwise the estimation error will quickly drive loss of tracking. While adding filtering can reduce the estimation error, this comes with its own cost through a slowing of the responsiveness of the controller. The right balance of these considerations is, of course, defined by the particular application and experimental aims.

Online estimation has been shown to be a powerful tool for providing position estimates to the tracking controller, and a well-designed and well-tuned feedback controller can often overcome the errors introduced by limited-quality online estimation. These controllers act directly on the measurement, comparing it to the expected (or desired) value and acting to reduce that error. If the estimator is unbiased, then on average, the controller drives that error to zero. If the estimator is biased it is still possible to drive the error to zero if in each cycle of the controller that error is reduced [163]. In addition, this feedback provides additional robustness to the measurement noise, disturbances (both external and from estimation error), and modeling error (such as changes in the actuator dynamics across their workspace) that are always present in any physical system. As noted above, the need to make the estimator computationally cheap necessitates making simplifying assumptions. This can reduce performance as those assumptions do not hold exactly in practice. For example, the linearization of the observation model results in a mismatch between the actual optical model and the model used for estimation [163, 38]. This mismatch increases as the particle moves from the point of linearization, systematically increasing the bias of the online estimates [163]. By contrast, offline estimation can apply algorithms of much higher computational complexity which are not limited to causal estimation, allowing the entire set of data to be used in making estimates at every point in time. To the best of our knowledge, only two groups currently use offline estimation. The first is the work in [38], where the online estimates used for tracking were numerically unbiased after tracking had concluded to improve the overall localization accuracy. The second is in the signal maximization approach [49, 141]. Because this approach reacts directly to the intensity, there is no need for online estimation; in addition, the pattern-less trajectory is ill-suited to many of the estimation approaches noted above. To handle the particular stream of data produced by this approach, an algorithm that combines nonlinear filtering with an optimization-approach to jointly estimate particle location and model parameters was developed [155, 49].

5.2. Fisher Information and Cramér-Rao Lower Bounds

Regardless of the estimation method employed, there is a fundamental limit to the achievable precision in parameter estimation. This minimum achievable estimator error variance for any unbiased estimator is known as the Cramér-Rao Lower Bound (CRLB) (see, for example, [164, 165]). The CRLB is determined from the Fisher information matrix FIM, defined by

FIM = E [{(\frac{\partial ln (p (y ∣ θ))}{\partial θ})}^{T} (\frac{\partial ln (p (y ∣ θ))}{\partial θ})],

(1)

where ln(p(y∣θ)) is the log-likelihood function of the recorded signal y and θ is the vector of parameters one wants to infer. The CRLB is then given by the corresponding diagonal elements of the inverse of the FIM:

Var (θ_{i}) \geq [F^{- 1}]_{i i} .

(2)

Understanding the CRLB for a particular experimental setting can help in gauging the quality of a particular experiment. Of course, as this is a theoretical limit, practical estimators typically perform worse than the CRLB so it should not be interpreted as the variance in a specific experiment but as a guide as to what can be achieved. Note that the use of MLE is in part justified because such estimators achieve the CRLB in the limit of large amounts of data.

While the CRLB directly gives the lower bound on the achievable precision, the FIM can be understood as representing the information content contained in the measurements. A “larger” FIM (in an appropriate sense) implies a larger amount of available information. This can translate into being able to meaningfully measure smaller changes in the parameter of interest. Similarly, increasing the number of parameters to be estimated will increase the CRLB for any one of the parameters; in essence there is a finite amount of information in the data and increasing the number of estimated parameters “spreads” that information over a larger set. As a result it is always better to directly measure as many parameters as possible to ensure the highest quality estimation in the parameters that matter.

The first comprehensive application of FIM in microscopy was by Ram et al. [144] where the framework was applied in a consistent and detailed way to derive the CRLB for particle localization in widefield superresolution microscopy (a comprehensive tutorial on Fisher information for optical microscopy published by the same research group [146] is a good source for further information). The FIM has found utility as an analysis tool for comparing different widefield microscopy techniques [166] and as a design tool [167]. It has also been applied in the RT-FD-SPT setting, beginning with the work of Gallatin and Berglund [145] where they showed that the FIM for particle localization based on laser scanning depends on the electric field gradient of the laser: increasing this gradient increases the FIM and thus decreases the CRLB, implying the ability to do better estimation. It has more recently been used as a design tool in, e.g., designing illumination patterns [38, 37] and optimizing the design of the tracker system [50], as well as a means of comparing RT-FD-SPT performance [31].

When using the CRLB, it is important to verify the assumption that the estimator is unbiased. As an example, consider estimating the position of a particle along a line from measurements of the intensity using a simple difference-over-sum estimator. In Fig. 7a we show the the results of 10,000 Monte Carlo runs of such an estimator (see Supporting Information for more details). The plot shows the estimator mean (solid black line), the realized estimator uncertainty (±3σ, purple region), and the theoretical lower bound ( $\pm 3 \sqrt{CRLB}$ , dotted black line). Near the center, where the mean follows the true position (indicating that the estimator is unbiased), the estimator uncertainty is larger than the CRLB, as expected. At the edges, however, there is a clear bias in the estimator as it approaches a constant value. As a result, the uncertainty drops to zero and the CRLB is no longer a relevant guide.

Figure 7: — Understanding the CRLB. (a) Results of 10,000 simulations of difference-over-sum estimation as a function of particle position. Black line: estimator mean, showing clear bias away from the center; purple region: ±3σ where σ is the standard deviation in the estimates at each particle position; dotted black line: $\pm 3 \sqrt{C R L B}$ . (b) The CRLB for the position of a particle in the plane using a 3-point measurement constellation (solid black dots), shown as an ellipse on a grid of particle locations. The major and minor axes of each ellipse are given by the largest and smallest eigenvalues, respectively, of the inverse of the FIM and demonstrate the dependence of the CRLB on the parameter values (see Supporting Information for more details). In both figures, distances are normalized to the beam diameter ω and are thus unitless.

It is also important to understand how the FIM changes with the true value of the parameters. To illustrate this, Fig. 7b shows a visualization of the CRLB for estimating particle location as a function of particle position in the plane based on a 3-point measurement constellation (see Supporting Information for more details). The major and minor axes of each ellipse are defined by the maximum and minimum eigenvalues, respectively, of the inverse of the FIM given that the true particle location is at that point. Note that smaller ellipses indicate a lower CRLB and thus (possibly) increased precision in the estimate. Behind each measurement point the achievable precision is quite poor. The CRLB improves as the particle moves to points farther from, and symmetric to, the measurement locations. Such visualization can be immensely helpful when designing or evaluating a particular RT-FD-SPT system [31, 50].

6. Calibration and Characterization

In order to maximize the performance of a given RT-FD-SPT system, and to understand its limitations, it is vital that the system is calibrated and characterized. Calibration is, of course, a natural component of the workflow in optical microscopy and is typically focused on understanding the actual PSF of the instrument and on determining the spatial resolution. The dynamic nature of RT-FD-SPT requires a more complex characterization that includes the additional steps of calibrating the actuation axes with respect to the optical coordinate system, finding the static localization precision, and determining the dynamic tracking performance. At each of these steps, adjustments to the hardware and tuning of algorithm parameters may be made; these changes often impact other elements of the systems and thus calibration and characterization is typically done in an iterative fashion as a system is tuned for a particular experimental aim.

Characterizing the PSF of the specific system being used is as important in RT-FD-SPT as in standard optical microscopy. Accurate knowledge of its shape can help in selecting algorithm parameters, such as the orbital radius for single-detector scanning methods [168, 145, 50], and is a key element when determining the minimum uncertainty achievable in particle position estimation [144, 145, 146, 50]. There are well-known techniques for PSF characterization (see, e.g., [169]) with the essential idea being to fix a small fluorescent particle like a fluorescent nanosphere or a quantum dot to a coverslip, acquire fluorescence intensity measurements from a 3D raster scan of the excitation light beam across the particle, and fit this data to an experimental model. It is important that the dimensions of the fluorescent particle are significantly smaller than those of the focused light beam so that it may be considered a point source of fluorescence that has a negligible effect on the measured PSF of the optical system. To avoid the effect of the coverslip on the axial component of the PSF, the fluorescent particle is typically embedded in a polymer or gel when characterizing the axial component of the PSF [170]. Alternatively, assuming the PSF to be symmetric about the XY plane, a substrate-bound particle can be used to determine the axial component only in the direction of the laser beam (away from the coverslip). To limit spherical aberration or broadening of the laser focus due to refractive index mismatch between the immersion medium and the sample, it is advisable to use a water-immersion or water-dipping objective.

In general, the optical axis of a RT-FD-SPT system will not be exactly aligned with the vertical axis of the actuators. This misalignment creates systematic errors in estimation if it is not known and can lead to vertical motion being misinterpreted as dynamics in the lateral directions [163]. An example of this is shown in Figure 8a where this effect was demonstrated on an orbital tracking RT-FD-SPT microscope [163]. This misorientation can be easily determined from the PSF characterization data and represented as a simple 3D rotation [49] (Figure 8b). Alternatively, if the beam and a sample stage can be scanned independently, one could use a fluorescent particle and move it along each actuator axis while doing a 3D raster scan, an orbital scan, or signal maximization. As this calibration need is unique to tracking microscopes, it represents an area requiring further development.

Figure 8: — Examples of calibration and characterization. (a) Characterization of actuator to optical axes alignment (adapted with permission from [163] © The Optical Society). (b) PSF characterization showing rotation with respect to actuator axes (adapted with permission from [49] © The Optical Society). (c) Fluctuations of tracking system to a static particle (reprinted from [75]. © 2019 Wiley-VCH). (d) Step response of system (adapted with permission from [56] © 2010 American Chemical Society). (e) Tracking helical trajectory (adapted with permission from [56] © 2010 American Chemical Society). (f) Tracking free diffusing particle (adapted with permission from [49] © The Optical Society). (g) Distribution of measured diffusion coefficients (top) and tracking duration of freely diffusing particles (bottom) (adapted with permission from [110] © 2012 American Chemical Society).

Once the PSF has been characterized, the static localization performance of the system can be investigated. This performance is affected by several factors, including the PSF of the instrument, the ability of the system to lock onto a fixed particle, and the steady-state behavior of the algorithm when “tracking” a fixed particle (Figure 8c). These elements can be characterized using the same physical setup as that for characterizing the PSF, namely a particle fixed to a coverslip (or embedded in a polymer or gel). Using this method, Hou and Welsher showed that the controller gains have a large effect on the size of these fluctuations [20], emphasizing the need for good tuning of controller parameters. Note that there is a balance to be achieved here: selecting gains that are too aggressive can lead to large fluctuations that can increase the variance of the position estimates or even result in loss of tracking, while selecting them too conservatively can lead to lack of sensitivity and an inability to following moving particles.

The final piece in calibration and characterization is studying the tracker performance in dynamic tests. These tests can be collected into four categories that make different trade-offs when weighing the need to know ground truth against the biological relevance of the test conditions: deterministic motion, simple diffusion, engineered samples, and synthetic motion. In deterministic motion, a particle is once again fixed to a coverslip. The coverslip is then moved in a known and controlled way using, e.g., a nanopositioning stage. Piezoelectric-based nanopositioning stages are tested and calibrated by the equipment manufacturers using high precision laser interferometers and typically provide nanometer or better positioning accuracy and precision, making them excellent choices for generating the (known) reference motion. Typical patterns include steps [28, 24, 150, 151, 19, 37] ((Figure 8d), circles [56], sinusoids [28, 18, 67, 20, 38], and helices [56, 19, 44] (Figure 8e). Because the ground truth of the motion is known, this approach can be used to characterize response time and tracking error. However, these methods are not representative of the motion of a particle in an aqueous or cellular environment and do not allow for the study of performance with respect to motion parameters such as diffusion coefficients.

Simple diffusion is a well-accepted model for describing the dynamics of biological macromolecules in their native environments. In this second category, particles of known size are placed in an aqueous environment of known viscosity (Figure 8f). The diffusion coefficient, governed by the well-known Stokes–Einstein relation, can be selected by controlling the particle size, the viscosity of the solution, and the temperature. This method has been performed using fluorescent microspheres [51, 163, 69, 151, 171, 23, 43, 75], “giant” quantum dots [54, 20], and regular quantum dots [53, 21, 109, 34, 35, 139, 82, 49] diffusing in solutions of agarose [28, 49], sugar-water [65, 69], glycerol-water [18, 24, 53, 51, 109, 33, 66, 56, 34, 35, 139, 110, 163, 172, 171, 43, 44, 49, 95], and water [21, 20, 75, 95]. After a tracking experiment, the diffusion coefficient and localization error can be estimated from the localizations using, e.g., a mean squared displacement (MSD) analysis [173] or a Maximum Likelihood Estimator (MLE) [148] and statistics on performance gathered by repeating multiple experiments (Figure 8g). While this motion is biologically relevant, there is no ground truth on the particle trajectory and even the “known” diffusion coefficient is inexact due to factors that include dispersion in the sizes of manufactured particles and local variations in the viscosity and temperature of the sample.

The third category is similar to simple diffusion but rather than using a simple aqueous solution, the particles are placed in an engineered environment that increases the complexity of the particle dynamics [174]. Researchers have used oil–water interfaces [175], and proposed engineered lipid membranes, mesoporous materials, opals, and polymer solutions [174]. While this approach is more difficult to set up, the particle motion may be more representative of a situation of interest. As with simple diffusion, however, ground truth is essentially impossible to come by.

The final category is a method called synthetic motion [174, 176, 177]. As with the method of deterministic motion, this approach uses a fluorescent particle fixed to a coverslip and moved by a nanopositioning stage. Instead of a simple, deterministic trajectory, however, the stage is used to realize almost any biologically relevant process, including simple diffusion, tethered motion, confined diffusion, hop diffusion, and more. Because these motions are realized artificially, ground truth is known on trajectory and motion model parameters. However, the local environment remains simple, without confounding factors such as background fluorescence from nearby structures, and creating accurate realizations of the desired motion, particularly for fast dynamics, can be challenging and is ultimately limited by the capabilities of the nanopositioning stage being used [177].

As RT-FD-SPT microscopes add multi-color detection, contextual imaging, and spectroscopy, it is important to calibrate and characterize each additional function and to ensure the co-alignment of each element in the system. A major challenge is that of unintentional motion resulting from thermal expansion, stage drift, and other sources. While the tracking controller is robust against these unintentional motions and will remain locked onto the particle of interest, these unintentional motion sources will affect any position or dynamic parameter estimation. To mitigate this and to achieve nanometer precision over long durations, the stage holding the sample can be stabilized with active feedback. This can be achieved in multiple ways including imaging of gold nanorods [77], using fiducials [178], and image correlation [179], and commercially available systems that use back-reflection of infrared light off of the coverslip, to provide a signal for the control to hold steady, thereby providing a stable global frame of reference. Aligning spectrally separated detection channels faces the same challenges as in other microscopes, and commercially available fluorescent microspheres with multiple different fluorescent dyes can be used for chromatic registration. Similarly, NIST traceable fluorescent samples enable calibration for fluorescence intensity and wavelength [180]. Finally, it is generally useful to obtain a contextual image simultaneously with the tracking. To register these two data streams, one must learn a transformation between the camera and tracking channel coordinates. This can be achieved by imaging with both channels a sample that has multiple fluorescent beads across the entire field of view, and then fitting a transformation between both images.

7. Tracking Performance

The performance of RT-FD-SPT is a challenging topic to cover. This is due to the stochastic nature of RT-FD-SPT as well as inherent trade-offs between different aspects of performance. There is no single metric that completely describes the behavior of RT-FD-SPT systems, and it is important to consider these techniques with respect to many different factors, each of which may have different relevance in different applications. Additionally, reporting of system performance has not been consistent, and existing methods have not necessarily been pushed to their limits, making direct comparisons between different systems difficult. In this section, we first cover the currently reported performance metrics. Following this we describe a set of practical performance metrics with the goal of guiding a researcher interested in applying RT-FD-SPT in selecting an approach. One of the challenges is knowing the ultimate performance of a system. A promising approach is to analyze the system performance from the fundamental statistics of the tracking process and the resulting trade-offs. Finally, we discuss performance fit to experimental aims.

7.1. Current Performance

Several reported metrics have been used to quantify the performance of RT-FD-SPT methods, including localization precision, fastest tracked diffusion coefficient, longest tracking duration, lowest photon count rate, and experimental tracking precision. These can be lumped into two broad categories: tracking precision and tracking success.

For precision, various theoretical metrics have been used. The Fisher information and CRLB provide an indicator of the precision with which a static particle can be localized, typically as a function of the number of detected photons [145, 38, 31, 50, 37]. This is useful since ideally the feedback is fast enough that the particle can be considered to be approximately static during one cycle. Another performance metric is the mean-squared tracking error, which can be predicted theoretically using suitable approximations [168] or using a numerical simulation [136, 22, 168, 181, 29, 162]. The precision of an RT-FD-SPT system depends on many factors. The CRLB is of course influenced strongly by the number of detected photons and the signal-to-background ratio, but also by the way in which the position is measured. Optimal measurements can have a significant impact on the precision [145, 38, 31, 37]. Also, in general, a decrease in the detection volume size corresponds to a lower CRLB (and thus increased precision). The simulated mean tracking error is dependent on the CRLB but also depends on the estimation and control algorithms and the system model.

Metrics that focus on the “success” of the tracking include the expected tracking duration [25], the percentage of particles lost within the simulation run time [162], and the largest diffusion coefficient that can consistently be obtained for some length of time [136, 22, 181, 162, 31]. The tracking success strongly depends on the feedback bandwidth of the overall tracking system, the size of the detection volume, and the rate of photon detection. As one would expect, broader bandwidths and larger detection volumes allow for faster particles to be tracked and for longer duration. In contrast, higher precision requires a smaller detection volume [31]. There is, therefore, generally a tradeoff between the precision and success of tracking. For this reason, one cannot simply combine these metrics into a single performance metric, as different applications might favor either speed or precision.

To compare different methods’ experimental performance, it is necessary to consider the interplay between different metrics. In Figure 9 we show the performance of different approaches with respect to four key metrics: diffusion coefficient, tracking error, tracked time, and photon count rate. Instead of simply considering the best reported values, we based the comparison on ranges of reported values to illustrate the regions of parameter space where the methods are known to work well (see Supporting Information for details). It is possible that each method can be applied outside of its known region, but such results have not yet been demonstrated, to our knowledge. There are, of course, also other metrics that could be considered, for example tracking depth, maximum tracking range, or metrics related to spectroscopic measurements such as fluorescence lifetimes (see Section 7.3). The ones shown here are what we consider to be the most generally applicable.

As a reference point, we also show one example of the reported performance of one conventional image-based 3D-SPT technique, namely the use of the double-helix point spread function (DH-PSF) [182]. We stress that this is simply a reference and not meant to represent the best performance of this technique, and this review does not intend to compare the performance of RT-FD-SPT and conventional image-based SPT approaches. The metrics shown here are not the ones most commonly used to evaluate conventional SPT performance and therefore are often not all reported, making a direct comparison between image-based and RT-FD-SPT performance non-trivial. This is, however, a promising avenue for further research. In any case, as discussed in Section 2, image-based and RT-FD-SPT are better thought of as complementary rather than competitive techniques.

As expected, in all metrics, all techniques show improved performance under easier conditions (i.e., slower diffusion values, higher count rates, etc.). Because cost is also a significant factor to take into account when selecting a method, the inset plots in Figure 9 show the performances normalized by the hardware cost shown in Table 1. These calculated costs are based on the most expensive components in the experimental realizations of the methods, such as the laser source, photodetectors, control hardware and software, and beam scanning components (details on these cost estimates can be found in the Supporting Information). We note that it may be possible to achieve comparable performance at a lower cost through careful choice and optimization of equipment.

Table 1:

Approximate cost of 3D RT-FD-SPT experimental setups in US Dollars

Method	Cost (USD)
TSUNAMI	195k
MINFLUX 3D	170k
Optical Cantilever	97k
3D-DyPlot	95k
Orbital 3D Highspeed	83k
Tetrahedral	83k
Quadrant	82k
Local Imaging	82k
Orbital 3D with tunable lens	55k
Extremum Seeking	55k

Open in a new tab

There are a few clear trends that emerge in Figure 9. For example, the comparison between the tracked diffusion coefficient and tracking error clearly indicates the trade-off between tracking speed and localization performance. While different techniques approach this trade-off in different ways, these results indicate you must give up on one to gain in the other. As mentioned above, this is the main reason why these metrics cannot simply be combined into a single, general measure of performance. Similarly, there is a clear decrease in tracking error with increasing photon count rate in all methods. Interestingly, while tracking time generally increases with decreasing diffusion coefficient, there is less of a trend in this comparison, possibly reflecting that factors other than loss of tracking led to the end of an experimental run. Comparison of the methods shows that those that implement fast beam scanning (MINFLUX, orbital scanning, and 3D-DyPLoT) have the better reported performance, but also have high cost and complexity. MINFLUX has a clear edge in precision, but lags when it comes to tracking duration. TSUNAMI has good performance, but appears to depend on very high photon count rates. TSUNAMI and MINFLUX are, however, significantly more expensive than the other methods (in the case of TSUNAMI this is due to the use of a femtosecond laser) and thus their position worsens in the cost-adjusted plots. Conversely, the extremum seeking method’s minimalistic hardware requirements make it inexpensive, and thus it has excellent performance-per-dollar.

7.2. Fundamental Performance and Trade-offs

The characterization of RT-FD-SPT performance has historically focused on practical and measurable performance. This is very useful but can make it difficult to compare systems, as each system needs to be tested in the same way and to the limits of its performance, which can be time-consuming and offers no guarantee that the best performing configuration of that particular system will be found. Additionally, experimentally measuring the performance does not provide a systematic way to design and improve these systems. A different approach is to look for fundamental concepts that underlay and unify RT-FD-SPT performance, and explain the experimentally observed trends in localization precision, tracking error, and tracking duration. The two fundamental ideas underlying these performance metrics are trackability and noise propagation.

The aim of quantifying trackability is to understand how different aspects of the system enable or inhibit locking onto and tracking the particle for an extended duration. This can be seen as the theoretical counterpart to tracking success where there is a correspondence with both the experimentally derived maximum tracked diffusion coefficient and maximum tracked duration discussed in the previous section. The analysis of trackability has two equivalent points of view. The first notion of trackability focused on understanding the probability of not losing a particle for a finite duration and was analyzed for the cases of a particle escaping the detection volume [22], for a particle going into a dark state [109], and for photobleaching [110]. Using this framework, a maximum trackable diffusion coefficient was calculated [22]. The argument of this tracking probability gives a dimensionless number, $\frac{r}{\sqrt{D T}}$ , where r is the characteristic dimension of the localizable volume, D is the diffusion coefficient, and T is the integration time. Larger values of the dimensionless number indicate better trackability. It was then shown that tracking duration follows a geometric distribution related to the tracking probability [109]. The tracking probability directly relates to the mean duration of tracking. The second approach focuses on understanding the probability distribution of the time for a particle to first exit the detection volume given a fixed detection volume. The first moment of this distribution gives us the mean escape time [43, 141], which is the average time to lose a particle, and is proportional to $\frac{r^{2}}{D}$ . An effective controller increases the tracking duration, which increases the value of the proportionality constant [141]. A missing aspect of these analyses is the effect of photon detection. By considering both particle motion and photon detection, Berglund developed the dimensionless number $\frac{r^{2} Γ}{D}$ as a metric for trackability, where Γ is the mean detected photon rate [185]. Although research in this area is far from over, these results provide a basis for improving the duration of tracking in a systematic way.

The second fundamental aspect of performance is how noise propagates through the system. Both the motion of the molecule and the detection of photons are fundamentally stochastic processes and they are typically the dominant sources of noise. This has been characterized in two key ways. The first was the adoption of the Fisher information and CRLB [145, 146, 38, 31, 50]. These provide a basis of understanding the best possible precision with which a particle can be localized as well as a lower bound on the noise from photon detection and online estimation of position. Using these metrics as a tool for analysis, it was shown that smaller beam widths and a larger number of detected photons give better localization precision [145, 50], essentially reducing the impact of stochastic variation in the photon signal. Additionally, different laser modes can give more precise localizations [38]. The second analysis approach uses the lens of stochastic dynamic systems to understand how diffusion and shot noise limit system performance [168, 185, 183]. It was shown that tracking error follows a $\sqrt{D}$ trend. Additionally, there is a minimum tracking error that is proportional to localization uncertainty and the bandwidth of tracking and online estimation. Further, the photon detection rate sets a lower limit on the tracking error variance.

One of the most frustrating and interesting aspects of RT-FD-SPT is the emergence of parameters that are subject to a trade-off between localization uncertainty and tracking duration. The first parameter to consider is that of photobleaching. Photobleaching puts a limit on the expected number of photons one can use for the competing objectives of tracking and localization. Increasing laser power improves localization at each time point, but it also increases the probability of bleaching, which reduces tracking duration. Another parameter to consider is that of the beam diameter and scan pattern. It was shown that smaller beam diameters (and thus a smaller detection volume) improve localization uncertainty [145, 50] while increasing the detection beam diameter and volume increases the maximum trackable diffusion coefficient [22, 168], enabling locking onto fast diffusing particles for longer duration [21, 75]. One interesting route out of this trade-off stalemate comes from adaptation of the beam diameter or scan pattern in response to the particle’s diffusion coefficient, providing better localization for slow diffusing particles without sacrificing tracking duration of fast diffusing particles [75]. The final trade-off parameter is the system bandwidth [168, 185]. Long integration periods give time to collect more photons for localizing the particle, but also increase the likelihood of losing the particle, implying that there is an optimal bandwidth or integration time for a given weighting between these two competing phenomena.

7.3. Performance–Application Fit

Much of the testing and analysis of RT-FD-SPT systems was done in the context of determining a particle’s trajectory and inferring its dynamic model. In this experimental aim, both tracking duration and localization precision are important in determining the uncertainty in the estimates of the model parameters. However, an optimal position measurement does not, in general, correspond to optimal inference of model parameters. Similarly, in spectroscopy applications (Section 3.2), an optimal position measurement does not generally correspond to an optimal spectroscopic measurement. Determining the best approach for a specific application in general involves assessing four factors: the speed (e.g., the diffusion coefficient) of the particle of interest, the achievable photon detection rate under experimental constraints, the time scale of the process under investigation, and, if a separate beam is used for concurrent spectroscopy, the detection volume of this beam. Intuitively, the tracking error of a technique should be smaller than the detection volume of the spectroscopic beam to ensure uninterrupted and quality data. Additionally, experiments with slow processes or low detected spectroscopic photon rates will require longer duration tracking to attain a particular measurement uncertainty, while fast moving particles will need a high temporal resolution to maintain the particle in the detection volume of the tracking beam and, if used, of the spectroscopic beam. Finally, when long tracking durations are needed, the system will generally need a larger tracking range to follow the particle through a larger volume. With these considerations, selection of a technique can be guided at least in part by reported performance (Figure 9).

While these considerations help to guide a selection of technique, they are qualitative, and a formal and quantified approach to evaluating performance and fit to a specific application remains an unexplored aspect of RT-FD-SPT spectroscopy. This represents an open challenge for future development. A promising approach is to develop expressions for the Fisher information and CRLB for a particular experimental aim. This has been done in the case of FRET and charge-transfer techniques [157, 158]. When this is done in the context of RT-FD-SPT, a connection between experimental aim, relevant system parameters, and performance metrics is established, enabling an assessment of application-specific performance. A design that optimizes this relationship may end up being different from the optimal design for position measurement.

8. Future Perspectives

As demonstrated in this review, RT-FD-SPT research has been productive, leading to several different methods, and clearly demonstrating the potential of these techniques for biological research. In this section, we discuss our envisioned future of the field with regard to research into both the technique itself and applications. We remark on the trends of RT-FD-SPT, development challenges and opportunities, and the adoption in the wider scientific community.

8.1. Current and Future Trends

Research into RT-FD-SPT methods continues and recent work shows several promising trends. One example is an improvement in performance over time, especially with regard to tracking error (see Figure 9). A few very recent studies indicate interest in, and demonstrate the importance of, developing a theoretical understanding of RT-FD-SPT, particularly with regard to statistical analysis of precision [31, 50, 113]. Similarly, developing a fundamental understanding of the elements that determine how long particles can be tracked has led to a general increase in the duration of tracking. As our knowledge of the fundamental trade-offs between tracking precision and tracking duration continues to improve, researchers are designing their systems with these trade-offs in mind. This has enabled longer tracking duration with only minor increases in position estimation uncertainty. These recent developments that leverage the statistical basis of performance are expected to grow and to extend to other important aspects of RT-FD-SPT such as control design and tailoring a system for a specific experimental aim.

There are also promising developments in the controllers applied in RT-FD-SPT. Controller design has begun to move away from simple, PID-based tracking controllers toward more advanced techniques such as optimal, nonlinear, and adaptive control. Two key drivers for these trends are that the problem of tracking a diffusing particle is fundamentally a stochastic one, and that the experimental context changes over time. The next generation of RT-FD-SPT methods will need to strike a balance between enhancing performance through advanced control techniques, and ensuring ease of implementation to promote adoption. This tension in that balance provides unique challenges and opportunities for designers.

Another set of trends in RT-FD-SPT is advances in the design of the excitation and detection subsystems. These trends can be broken down into increasing the tracking performance of the system, understanding the experimental context, and overcoming the limitations of fluorescence-based tracking. A promising direction looks at real-time optimization of the laser illumination intensity to improve the performance of the system. This was first done by McHale et al. [21] for the tracking duration, and by Balzarotti et al. [38] in MINFLUX. This trend of adjusting or engineering the illumination to meet the experimental needs is likely to continue and to produce additional performance gains. The engineering and optimization of the illumination pattern and detection is a largely unaddressed opportunity.

As experimental aims move beyond just tracking the particle, complementary information must also be acquired. This is first seen as the trend toward the inclusion of various spectroscopic methods to infer more than just the particle’s position and dynamics, measuring a more complete state of the particle. Some of the emerging experimental aims require estimating the particle’s relative position to some other particle or feature. This could be the folding of proteins or binding onto a specific receptor. To meet these needs, there has been a trend of adding additional detection channels. One form of this is the addition of multi-color detection allowing for localization of two separate fluorescent labels [45, 115]. Some of the emerging experimental aims include understanding the particle’s behavior in the context of the cellular environment such as proximity to the cell membrane. To meet this need, there has been research focusing on including simultaneous imaging of the cellular environment [82, 186]. The trend of adding parallel co-aligned measurements and imaging modalities will only continue as the experimental aims move beyond observing the motions of a single particle.

One trend we may see is the development of tracking methods that do not use fluorescence such as interferometric scattering (iSCAT) [187, 188]. iSCAT considers the interference between the light scattered by the particle of interest and the portion of incident light that is reflected by a nearby interface. It therefore represents a different source of photons than fluorescence. Although iSCAT has already proven to be a powerful imaging technique, it has only been used once with real-time feedback, in the form of the ABEL trap [189]. iSCAT overcomes many of the trade-offs of fluorescence-based tracking as discussed in Section 7, as saturation and photobleaching can be minimized by choosing a wavelength of light that is negligibly absorbed by the tracked particle. This makes it a very attractive technique for RT-FD-SPT. A recent study [31] showed that iSCAT should perform better for some samples (particularly relatively large ones), even at the same illumination intensity. Seeing as a much higher illumination intensity (~ 1000 times higher) is typically used for iSCAT with no phototoxic effects, the method should, in fact, offer excellent performance for a wide range of samples. A major drawback of iSCAT, however, is that it is very challenging to use in vivo. This is firstly due to the crowded molecular environment: as all molecules scatter light, iSCAT fails to provide the same selectivity offered by fluorescent labeling, giving rise to significantly enhanced background that severely compromises the signal-to-background ratio. Secondly, iSCAT typically requires a particle to be close to a reflective interface, as the method requires coherence between the scattered and reflected light fields.

8.2. Open Challenges and Opportunities

As the field of RT-FD-SPT pushes the boundaries of performance and application, open challenges and development opportunities emerge, both in individual components of the tracking system and in the combination of these components into a total system. We start this discussion with the opportunities in controller design. To date, high-bandwidth tracking implements very expensive beam scanning and stage scanning actuators. There is a need to develop affordable high-bandwidth actuation systems through refining their design or through improved control. As bandwidth increases, the limiting factor for tracking will transition away from actuator limitations and toward photon detection, opening up opportunities in considering more complex tracking controllers. We have seen that the application of adaptive, optimal, and nonlinear controllers improves overall tracking performance over traditional PID feedback approaches. Further development of adaptive control approaches to enable the system to react in real-time to changes in the motion of the tracked particle, changes in the experimental context, and changes in the detection of photons promises exciting advances in technique and application. While there have been some initial efforts in this area, there remains significant opportunities to use adaptation in the face of uncertainties and error arising from measurement noise, modeling error, motion blur, and other sources. The use of adaptation shifts the paradigm of RT-FD-SPT to greater levels of autonomy, enabling single molecule experiments to transition from single measurements to large data sets. Similarly, if the initialization of tracking could be automated and optimized, overall throughput and utility would be significantly enhanced.

Similar to controller design, there are opportunities and open challenges for estimation in RT-FD-SPT. To date, one of the main goals for online estimation was that it must be easy to implement and fast to compute. This has produced estimators that show increased error as the particle moves away from the center of the detection pattern. There is an opportunity to implement online position estimators that have lower error over a larger area of the detection pattern and to couple these estimators more directly to new, adaptive controllers. An additional limitation of current approaches is that the noise from photon detection is typically not filtered or taken into consideration in the control loop. A few groups have used Kalman filters to help mitigate the impact of this noise; however, the adoption of filtering should be routine. Another possible avenue for research concerns the photon statistics itself. Thus far, when photon noise has been taken into account in estimators for RT-FD-SPT, it has generally been assumed to be Poisson-distributed. However, single emitters exhibit sub-Poissonian noise [190]. To achieve the ultimate performance in tracking single molecules at low photon count rates will likely require taking this into account. It is particularly important as RT-FD-SPT systems move toward single photon estimation and control where accurate noise models and filtering become exceedingly important. Finally, it would be of significant interest to compare the performance of RT-FD methods with that of more traditional, camera-based techniques in applications where the capabilities of the different methods overlap. Understanding and comparing how these different experimental techniques handle noise and estimation error, and how these errors affect the final parameter estimation, is important in evaluating methods when determining what is most appropriate to address an experimental aim.

The development of RT-FD-SPT microscopes has largely focused on using Gaussian lasers in combination with a scan pattern, or a set of detectors. While a Gaussian laser is practical and ubiquitous, it is unclear if it is optimal for particle tracking. MINFLUX has demonstrated that other laser modes and specially engineered beam profiles can provide better localization precision [38]. The optimization of the laser illumination and detection for tracking performance or to fit the experimental aim is still an open challenge and questions related to understanding the benefits and weaknesses of alternate illumination approaches are largely unexplored. As research emerges to address this, it will be important to develop methods and performance metrics that allow researchers to understand how to select the illumination and detection patterns that best fit their application and experimental aim.

Finally, the quantification of total system performance is an area that is still evolving. Understanding the limits of a particular system is still challenging to assess, and a direct comparison between systems is very difficult. Trackability is the first measure of how different systems compare; however, it is just a start. Its main weakness is not taking into consideration the bandwidth of the controller. Additionally, application-specific metrics (e.g., related to fluorescence lifetime measurements) are needed to quantify how a particular system and configuration will meet different experimental aims. A selection tool that utilizes these (yet-to-be-developed) application-specific metrics to enable a selection of the RT-FD-SPT methods that best fits a particular experimental aim would be a valuable contribution to the community.

8.3. Adoption of RT-FD-SPT

The final challenge in RT-FD-SPT is driving awareness and adoption of these methods. The development of the core technologies of RT-FD-SPT is well established, with multiple techniques available for the simultaneous tracking and spectroscopic measurements of single particles and molecules in their native environments. We believe that these methods represent a valuable addition to the toolbox of researchers and that they enable a vast array of research questions to finally become addressable. Unfortunately, there are still barriers to adoption that must be overcome to make these techniques more widespread. One important consideration is the need to be able to reproduce a specific experimental setup and a particular experiment. Currently, the descriptions of experiments in most published works are not detailed enough to be re-created in another laboratory without significant additional experimentation. In many cases, while the hardware for a method may be relatively simple, the software development is quite complex and methods for parameter tuning are often poorly described. Many systems rely on proprietary tools such as NI LabVIEW rather than open-source hardware, making it difficult to share code. A detailed protocol paper or open-source software approach would go a long way to making the field accessible to newcomers or non-specialists. Another barrier to adoption and development is the cost of the RT-FD-SPT microscope. As shown in Table 1, many of the techniques discussed in this review are expensive to implement. Additionally, it may be unclear to a newcomer which aspects of a setup are essential or worth spending more money on and which can be achieved using a lower-cost part. While we hope this paper helps address that somewhat, often the determination of what is important depends on the specific application and experimental conditions. Finally, the inter-disciplinary nature of RT-FD-SPT can be a significant hurdle. Many of the research groups engaged in RT-FD-SPT have teams that span biology to electrical engineering. The development of open-source hardware and software will go a long way to enabling the equitable adoption of RT-FD-SPT, and enabling various applications.

To date, RT-FD-SPT methods have been applied in a handful of applications and produced excellent results (see Section 2). We believe that with increasing awareness and adoption, many more biologists and spectroscopists will see the great potential of the technique to studying in real time any biological process that involves the translational motion of molecules, macromolecules and/or larger particles like viruses and vesicles. The technique has the potential to revolutionize many research fields, including those of virology and drug delivery [15]. We would especially like to see more spectroscopy applications. In the SMS field, there is a large push toward single-molecule studies in natural environments, and RT-FD-SPT is perfectly suited to this task. We believe it can be a widely used technique, alongside other advanced techniques such as superresolution microscopy and nonlinear microscopy. The combination of microscopic and spectroscopic measurements is potentially a powerful avenue of research and should be of special interest. We also encourage using RT-FD-SPT to revisit thousands of earlier studies, but now for (macro)molecules or particles in live cells as they explore various environments for longer times and probing their dynamics and interactions with improved spatiotemporal resolution.

Supplementary Material

supinfo

NIHMS1819935-supplement-supinfo.pdf^{(280.4KB, pdf)}

Acknowledgements

B.v.H. and N.A.V contributed equally to this work. S.B.A. acknowledges funding from NIH through grant NIGMS 5R01GM117039. T.P.J.K. acknowledges funding from the South African National Research Foundation through grant N00500/120387.

Biographies

graphic file with name nihms-1819935-b0002.gif

Bertus van Heerden received a B.Sc., B.Sc. (Hons.) and M.Sc. in Physics from the University of Pretoria in South Africa and is now pursuing a Ph.D. at the same university. His research focuses on single molecule spectroscopy of light-harvesting complexes and single-particle tracking.

graphic file with name nihms-1819935-b0003.gif

Nicholas A. Vickers started his career at California Polytechnic State University where he pursued a B.S. in Materials Engineering and B.A. in Physics, specializing in microfluidics, microfabrication, and micro-optics. He went on to serve as a process engineer at Soraa, followed by Technical Accounts Manager and Field Service Engineer at ASMPT. He is currently a Mechanical Engineering PhD candidate at Boston University, Boston, MA, USA. His current research focuses on information optimal control of tracking spectroscopy microscopes. His research interests include high-speed control systems for optical microscopes, information optimization, engineered point spread functions, and single-particle tracking.

graphic file with name nihms-1819935-b0004.gif

Tjaart P.J. Krüger received his Ph.D. in Biophysics from the Vrije Universiteit Amsterdam in The Netherlands and thereafter moved to South Africa to establish biophysics research at the University of Pretoria. He is currently an associate professor in physics at the same university. His research focuses mainly on single molecule spectroscopy and transient absorption spectroscopy of photosynthetic light-harvesting complexes, semiconductor nanostructured materials, organic polymers, and metallic nanoparticles.

graphic file with name nihms-1819935-b0005.gif

Sean B. Andersson received a B.S. in engineering and applied physics from Cornell University, Ithaca, NY, USA, an M.S. in mechanical engineering from Stanford University, Stanford, CA, USA, and a Ph.D. in electrical and computer engineering from the University of Maryland, College Park, MD, USA. He is currently a professor of mechanical engineering and of systems engineering with Boston University, Boston, MA, USA. His research interests include systems and control theory with applications in single-particle tracking, biophysics, scanning probe microscopy, and robotics.

References

[1].Dupont A, Lamb DC, Nanoscale 2011, 3, 11 4532. [DOI] [PubMed] [Google Scholar]
[2].Chenouard N, Smal I, De Chaumont F, Maška M, Sbalzarini IF, Gong Y, Cardinale J, Carthel C, Coraluppi S, Winter M, Cohen AR, Godinez WJ, Rohr K, Kalaidzidis Y, Liang L, Duncan J, Shen H, Xu Y, Magnusson KE, Jaldén J, Blau HM, Paul-Gilloteaux P, Roudot P, Kervrann C, Waharte F, Tinevez JY, Shorte SL, Willemse J, Celler K, Van Wezel GP, Dan HW, Tsai YS, De Solórzano CO, Olivo-Marin JC, Meijering E, Nat. Methods 2014, 11, 3 281. [DOI] [PMC free article] [PubMed] [Google Scholar]
[3].Manzo C, Garcia-Parajo MF, Rep. Prog. Phys 2015, 78, 12. [DOI] [PubMed] [Google Scholar]
[4].Liu Z, Lavis LD, Betzig E, Mol. Cell 2015, 58 644. [DOI] [PubMed] [Google Scholar]
[5].Mathai PP, Liddle JA, Stavis SM, Appl. Phys. Rev 2016, 3, 1 011105. [DOI] [PMC free article] [PubMed] [Google Scholar]
[6].Shen H, Tauzin LJ, Baiyasi R, Wang W, Moringo N, Shuang B, Landes CF, Chem. Rev 2017, 117, 11 7331. [DOI] [PubMed] [Google Scholar]
[7].Von Diezmann A, Shechtman Y, Moerner WE, Chem. Rev 2017, 117, 11 7244. [DOI] [PMC free article] [PubMed] [Google Scholar]
[8].Zhong Y, Wang G, Annu. Rev. Anal. Chem 2020, 18 50. [DOI] [PubMed] [Google Scholar]
[9].Thompson MA, Lew MD, Badieirostami M, Moerner WE, Nano Lett. 2010, 10, 1 211. [DOI] [PMC free article] [PubMed] [Google Scholar]
[10].Zhong Y, Zhao L, Tyrlik PM, Wang G, J. Phys. Chem. C 2017, 121, 14 8023. [Google Scholar]
[11].Wang D, Wu H, Schwartz DK, Phys. Rev. Lett 2017, 119, 26. [DOI] [PubMed] [Google Scholar]
[12].Bouchard MB, Voleti V, Mendes CS, Lacefield C, Grueber WB, Mann RS, Bruno RM, Hillman EM, Nat. Photonics 2015, 9, 2 113. [DOI] [PMC free article] [PubMed] [Google Scholar]
[13].Moerner WE, J. Phys. Chem. B 2002, 106, 5 910. [Google Scholar]
[14].Cang H, Shan Xu C, Yang H, Chem. Phys. Lett 2008, 457, 4–6 285. [Google Scholar]
[15].Welsher K, Yang H, Faraday Discuss. 2015, 184 359. [DOI] [PubMed] [Google Scholar]
[16].Hou S, Johnson C, Welsher K, Molecules 2019, 24, 15 2826. [DOI] [PMC free article] [PubMed] [Google Scholar]
[17].Wehnekamp F, Plucińska G, Thong R, Misgeld T, Lamb DC, eLife 2019, 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
[18].Levi V, Ruan QQ, Gratton E, Biophys. J 2005, 88, 4 2919. [DOI] [PMC free article] [PubMed] [Google Scholar]
[19].Annibale P, Dvornikov A, Gratton E, Biomed. Opt. Express 2015, 6, 6 2181. [DOI] [PMC free article] [PubMed] [Google Scholar]
[20].Hou S, Lang X, Welsher K, Opt. Lett 2017, 42, 12 2390. [DOI] [PubMed] [Google Scholar]
[21].McHale K, Berglund AJ, Mabuchi H, Nano Lett. 2007, 7, 11 3535. [DOI] [PubMed] [Google Scholar]
[22].Andersson SB, Appl. Phys. B: Lasers Opt 2005, 80, 7 809. [Google Scholar]
[23].Lanzanò L, Gratton E, Methods Appl. Fluoresc 2014, 2, 2 024010. [DOI] [PMC free article] [PubMed] [Google Scholar]
[24].Cang H, Wong CM, Xu S, Rizvi AH, Yang H, Appl. Phys. Lett 2006, 88 223901. [Google Scholar]
[25].Enderlein J, Appl. Phys. B: Lasers Opt 2000, 71, 5 773. [Google Scholar]
[26].Berglund AJ, Mabuchi H, Opt. Express 2005, 13, 20 8069. [DOI] [PubMed] [Google Scholar]
[27].Cohen AE, Moerner WE, Opt. Express 2008, 16, 10 6941. [DOI] [PMC free article] [PubMed] [Google Scholar]
[28].Levi V, Ruan Q, Kis-Petikova K, Gratton E, Biochem. Soc. Trans 2003, 31, 5 997. [DOI] [PubMed] [Google Scholar]
[29].Wang Q, Moerner WE, Appl. Phys. B: Lasers Opt 2010, 99, 1–2 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
[30].Wang Q, Moerner WE, ACS Nano 2011, 5, 7 5792. [DOI] [PubMed] [Google Scholar]
[31].van Heerden B, Krüger TPJ, In Gregor I, Erdmann R, Koberling F, editors, Single Molecule Spectroscopy and Superresolution Imaging XIV. SPIE, 2021. 22. [Google Scholar]
[32].Shen Z, Andersson SB, Proc. Am. Control Conf 2009, 1682–1687. [Google Scholar]
[33].Shen Z, Andersson SB, Proceedings of the IEEE Conference on Decision and Control 2009, 6052–6057. [Google Scholar]
[34].Shen Z, Andersson SB, Proceedings of the IEEE Conference on Decision and Control 2011, 5856–5861. [Google Scholar]
[35].Shen Z, Andersson SB, IEEE Trans. Control Syst. Technol 2011, 19, 5 1269. [Google Scholar]
[36].Shen Z, Andersson SB, Mechatronics 2012, 22, 3 320. [Google Scholar]
[37].Zhang C, Welsher K, Entropy 2021, 23, 5 498. [DOI] [PMC free article] [PubMed] [Google Scholar]
[38].Balzarotti F, Eilers Y, Gwosch KC, Gynnå AH, Westphal V, Stefani FD, Elf J, Hell SW, Science 2017, 355, 6325 606. [DOI] [PubMed] [Google Scholar]
[39].Gwosch KC, Pape JK, Balzarotti F, Hoess P, Ellenberg J, Ries J, Hell SW, Nat. Methods 2020, 17, 2 217. [DOI] [PubMed] [Google Scholar]
[40].Müller BK, Zaychikov E, Bräuchle C, Lamb DC, Biophys. J 2005, 89, 5 3508. [DOI] [PMC free article] [PubMed] [Google Scholar]
[41].Dertinger T, Koberling F, Benda A, Erdmann R, Hof M, Enderlein J, In Nicolau DV, Enderlein J, Leif RC, Farkas DL, Raghavachari R, editors, Imaging, Manipulation, and Analysis of Biomolecules and Cells: Fundamentals and Applications III, volume 5699. SPIE, 2005. 219. [Google Scholar]
[42].Davis L, Sikorski Z, Robinson W, Shen G, Li X, Canfield B, Lescano I, Bomar B, Hofmeister W, Germann J, King J, White Y, Terekhov A, In Enderlein J, Gryczynski ZK, Erdmann R, editors, Single Molecule Spectroscopy and Imaging, volume 6862. SPIE, 2008. 68620P. [Google Scholar]
[43].Gremann JA, Davis LM, Opt. Express 2014, 22, 5 5641. [DOI] [PubMed] [Google Scholar]
[44].Perillo EP, Liu YL, Huynh K, Liu C, Chou CK, Hung MC, Yeh HC, Dunn AK, Nat. Commun 2015, 6, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[45].Liu YL, Perillo EP, Ang P, Kim M, Nguyen DT, Blocher K, Chen YA, Liu C, Hassan AM, Vu HT, Chen YI, Dunn AK, Yeh HC, ACS Nano 2020, 14, 7 7927. [DOI] [PMC free article] [PubMed] [Google Scholar]
[46].Masullo LA, Steiner F, Zähringer J, Lopez LF, Bohlen J, Richter L, Cole F, Tinnefeld P, Stefani FD, Nano Lett. 2021, 21, 1 840. [DOI] [PubMed] [Google Scholar]
[47].Dissanayaka KD, Canfield BK, Davis LM, Opt. Express 2019, 27, 21 29759. [DOI] [PubMed] [Google Scholar]
[48].Andersson SB, Appl. Phys. B: Lasers Opt 2011, 104, 1 161. [Google Scholar]
[49].Ashley TT, Gan EL, Pan J, Andersson SB, Biomed. Opt. Express 2016, 7, 9 3355. [DOI] [PMC free article] [PubMed] [Google Scholar]
[50].Vickers NA, Andersson SB, IFAC-PapersOnLine 2021, 54, 7 649. [DOI] [PMC free article] [PubMed] [Google Scholar]
[51].Cang H, Xu CS, Montiel D, Yang H, Xu S, Montiel D, Yang H, Opt. Lett 2007, 32, 18 2729. [DOI] [PubMed] [Google Scholar]
[52].Lessard GA, Goodwin PM, Werner JH, In Ultrasensitive and Single-Molecule Detection Technologies, volume 6092. SPIE, 2006. 609205. [Google Scholar]
[53].Lessard GA, Goodwin PM, Werner JH, Appl. Phys. Lett 2007, 91, 22 224106. [Google Scholar]
[54].DeVore MS, Stich DG, Keller AM, Ghosh Y, Goodwin PM, Phipps ME, Stewart MH, Cleyrat C, Wilson BS, Lidke DS, Hollingsworth JA, Werner JH, In Parak WJ, Osinski M, Liang X-J, editors, Colloidal Nanoparticles for Biomedical Applications X, volume 9338. SPIE, 2015. 933812. [Google Scholar]
[55].Dunlap MK, Ryan DP, Goodwin PM, Werner JH, Majumder S, Hollingsworth JA, Gelfand MP, Van Orden A, Appl. Phys. Lett 2020, 116, 2 21103. [Google Scholar]
[56].Juette MF, Bewersdorf J, Nano Lett. 2010, 10, 11 4657. [DOI] [PubMed] [Google Scholar]
[57].Spille JH, Kaminski TP, Scherer K, Rinne JS, Heckel A, Kubitscheck U, Nucleic Acids Res. 2015, 43, 2 e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
[58].Wang Q, Goldsmith RH, Jiang Y, Bockenhauer SD, Moerner WE, Acc. Chem. Res 2012, 45, 11 1955. [DOI] [PMC free article] [PubMed] [Google Scholar]
[59].Bespalova MI, Mahanta S, Krishnan M, Curr. Opin. Chem. Biol 2019, 51 113. [DOI] [PubMed] [Google Scholar]
[60].Sahl SJ, Leutenegger M, Hilbert M, Hell SW, Eggeling C, Proc. Natl. Acad. Sci. U. S. A 2010, 107, 15 6829. [DOI] [PMC free article] [PubMed] [Google Scholar]
[61].Sahl SJ, Leutenegger M, Hell SW, Eggeling C, ChemPhysChem 2014, 15, 4 771. [DOI] [PubMed] [Google Scholar]
[62].Jazani S, Sgouralis I, Pressé S, J. Chem. Phys 2019, 150, 11 114108. [DOI] [PubMed] [Google Scholar]
[63].So PTC, Ragan T, Gratton E, Carerro J, Voss E, In Farkas DL, Tromberg BJ, editors, Functional Imaging and Optical Manipulation of Living Cells, volume 2983. SPIE, 1997. 45–56. [Google Scholar]
[64].Peters IM, De Grooth BG, Schins JM, Figdor CG, Greve J, Rev. Sci. Instrum 1998, 69, 7 2762. [Google Scholar]
[65].Ragan T, Huang H, So P, Gratton E, J. Fluoresc 2006, 16, 3 325. [DOI] [PubMed] [Google Scholar]
[66].Hellriegel C, Gratton E, J. R. Soc., Interface 2009, 6, SUPPL. 1 S3. [DOI] [PMC free article] [PubMed] [Google Scholar]
[67].Katayama Y, Burkacky O, Meyer M, Bräuchle C, Gratton E, Lamb DC, ChemPhysChem 2009, 10, 14 2458. [DOI] [PMC free article] [PubMed] [Google Scholar]
[68].Reuel NF, Dupont A, Thouvenin O, Lamb DC, Strano MS, ACS Nano 2012, 6, 6 5420. [DOI] [PubMed] [Google Scholar]
[69].Dupont A, Gorelashvili M, Schüller V, Wehnekamp F, Arcizet D, Katayama Y, Lamb DC, Heinrich D, New J. Phys 2013, 15, 7 075008. [Google Scholar]
[70].Verdeny-Vilanova I, Wehnekamp F, Mohan N, Álvarez á. S., Borbely JS, Otterstrom JJ, Lamb DC, Lakadamyali M, J. Cell Sci 2017, 130, 11 1904. [DOI] [PMC free article] [PubMed] [Google Scholar]
[71].Donovan BT, Huynh A, Ball DA, Patel HP, Poirier MG, Larson DR, Ferguson ML, Lenstra TL, The EMBO Journal 2019, 38, 12 e100809. [DOI] [PMC free article] [PubMed] [Google Scholar]
[72].Marklund E, van Oosten B, Mao G, Amselem E, Kipper K, Sabantsev A, Emmerich A, Globisch D, Zheng X, Lehmann LC, et al. , Nature 2020, 583, 7818 858. [DOI] [PubMed] [Google Scholar]
[73].Lanzano L, Digman MA, Fwu P, Giral H, Levi M, Gratton E, Biophotonics J 2011, 4, 6 415. [DOI] [PMC free article] [PubMed] [Google Scholar]
[74].Mieskes F, Wehnekamp F, Plucińska G, Thong R, Misgeld T, Lamb DC, Data in Brief 2020, 29 105280. [DOI] [PMC free article] [PubMed] [Google Scholar]
[75].Hou S, Welsher K, Small 2019, 15, 44. [DOI] [PMC free article] [PubMed] [Google Scholar]
[76].Patel S, Ashwanikumar N, Robinson E, Xia Y, Mihai C, Griffith JP, Hou S, Esposito AA, Ketova T, Welsher K, Joyal JL, Almarsson Ö, Sahay G, Nat. Commun 2020, 11, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[77].Schmidt R, Weihs T, Wurm CA, Jansen I, Rehman J, Sahl SJ, Hell SW, Nat. Commun 2021, 12, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[78].Pape JK, Stephan T, Balzarotti F, Büchner R, Lange F, Riedel D, Jakobs S, Hell SW, Proc. Natl. Acad. Sci. U. S. A 2020, 117, 34 20607. [DOI] [PMC free article] [PubMed] [Google Scholar]
[79].Liu YL, Perillo EP, Liu C, Yu P, Chou CK, Hung MC, Dunn AK, Yeh HC, Biophys. J 2016, 111, 10 2214. [DOI] [PMC free article] [PubMed] [Google Scholar]
[80].Wells NP, Lessard GA, Phipps ME, Goodwin PM, Lidke DS, Wilson BS, Werner JH, In Single Molecule Spectroscopy and Imaging II, volume 7185. SPIE, 2009. 71850Z. [DOI] [PMC free article] [PubMed] [Google Scholar]
[81].DeVore MS, Stich DG, Keller AM, Cleyrat C, Phipps ME, Hollingsworth JA, Lidke DS, Wilson BS, Goodwin PM, Werner JH, Rev. Sci. Instrum 2015, 86, 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
[82].Welsher K, Yang H, Nat. Nanotechnol 2014, 9, 3 198. [DOI] [PubMed] [Google Scholar]
[83].Ha T, Biochemistry 2004, 43, 14 4055. [DOI] [PubMed] [Google Scholar]
[84].Alhadid Y, Chung SY, Lerner E, Taatjes DJ, Borukhov S, Weiss S, Protein Sci. 2017, 26, 7 1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
[85].Kaur G, Lewis JS, Van Oijen AM, Molecules 2019, 24, 3 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[86].English BP, Min W, Van Oijen AM, Kang TL, Luo G, Sun H, Cherayil BJ, Kou SC, Xie XS, Nat. Chem. Biol 2006, 2, 2 87. [DOI] [PubMed] [Google Scholar]
[87].Jiang Y, Douglas NR, Conley NR, Miller EJ, Frydman J, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 41 16962. [DOI] [PMC free article] [PubMed] [Google Scholar]
[88].Yildiz A, Tomishige M, Vale RD, Selvin PR, Science 2004, 303, 5658 676. [DOI] [PubMed] [Google Scholar]
[89].Dienerowitz M, Howard JA, Quinn SD, Dienerowitz F, Leake MC, Methods 2021, 193 96. [DOI] [PubMed] [Google Scholar]
[90].Krüger TP, Van Grondelle R, Physica B: Condense. Matter 2016, 480 7. [Google Scholar]
[91].Kondo T, Chen WJ, Schlau-Cohen GS, Chem. Rev 2017, 117, 2 860. [DOI] [PubMed] [Google Scholar]
[92].Gruber JM, Malý P, Krüger TPJ, Grondelle RV, Nanophotonics 2018, 7, 1 81. [Google Scholar]
[93].Mazal H, Haran G, Curr. Opin. Biomed. Eng 2019, 12 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
[94].Kapanidis AN, Margeat E, Ho SO, Kortkhonjia E, Weiss S, Ebright RH, Science 2006, 314, 5802 1144. [DOI] [PMC free article] [PubMed] [Google Scholar]
[95].Hou S, Exell J, Welsher K, Nat. Commun 2020, 11, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[96].Ha T, Ting AY, Liang J, Caldwell WB, Deniz AA, Chemla DS, Schultz PG, Weiss S, Proc. Natl. Acad. Sci. U. S. A 1999, 96, 3 893. [DOI] [PMC free article] [PubMed] [Google Scholar]
[97].Krüger TP, Ilioaia C, Johnson MP, Ruban AV, Papagiannakis E, Horton P, Van Grondelle R, Biophys. J 2012, 102, 11 2669. [DOI] [PMC free article] [PubMed] [Google Scholar]
[98].Schlau-Cohen GS, Yang HY, Krüger TPJ, Xu P, Gwizdala M, Van Grondelle R, Croce R, Moerner WE, J. Phys. Chem. Lett 2015, 6, 5 860. [DOI] [PubMed] [Google Scholar]
[99].Gwizdala M, Berera R, Kirilovsky D, Van Grondelle R, Krüger TP, J. Am. Chem. Soc 2016, 138, 36 11616. [DOI] [PubMed] [Google Scholar]
[100].Thyrhaug E, Krause S, Perri A, Cerullo G, Polli D, Vosch T, Hauer J, Proc. Natl. Acad. Sci. U. S. A 2019, 116, 10 4064. [DOI] [PMC free article] [PubMed] [Google Scholar]
[101].Van Dijk EM, Hernando J, García-Parajó MF, Van Hulst NF, J. Chem. Phys 2005, 123, 6. [DOI] [PubMed] [Google Scholar]
[102].Hildner R, Brinks D, Nieder JB, Cogdell RJ, Van Hulst NF, Science 2013, 340, 6139 1448. [DOI] [PubMed] [Google Scholar]
[103].Malý P, Gruber JM, Cogdell RJ, Mančal T, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2016, 113, 11 2934. [DOI] [PMC free article] [PubMed] [Google Scholar]
[104].Liebel M, Toninelli C, Van Hulst NF, Nat. Photonics 2018, 12, 1 45. [Google Scholar]
[105].Moya R, Kondo T, Norris AC, Schlau-Cohen GS, Opt. Express 2021, 29, 18 28246. [DOI] [PMC free article] [PubMed] [Google Scholar]
[106].Moya R, Norris AC, Kondo T, Schlau-Cohen GS, Nat. Chem 2022, 14, 2153. [DOI] [PMC free article] [PubMed] [Google Scholar]
[107].Haran G, Acc. Chem. Res 2010, 43, 8 1135. [DOI] [PubMed] [Google Scholar]
[108].McHale K, Mabuchi H, J. Am. Chem. Soc 2009, 131, 49 17901. [DOI] [PubMed] [Google Scholar]
[109].Wells NP, Lessard GA, Werner JH, Anal. Chem 2008, 80, 24 9830. [DOI] [PMC free article] [PubMed] [Google Scholar]
[110].Han JJ, Kiss C, Bradbury AR, Werner JH, ACS Nano 2012, 6, 10 8922. [DOI] [PubMed] [Google Scholar]
[111].Liu C, Obliosca JM, Liu YL, Chen YA, Jiang N, Yeh HC, Nanoscale 2017, 9, 17 5664. [DOI] [PMC free article] [PubMed] [Google Scholar]
[112].Chen Y-I, Chang Y-J, Nguyen TD, Liu C, Phillion S, Kuo Y-A, Vu HT, Liu A, Liu Y-L, Hong S, Ren P, Yankeelov TE, Yeh H-C, J. Am. Chem. Soc 2019, 141, 40 15747. [DOI] [PMC free article] [PubMed] [Google Scholar]
[113].Tan X, Welsher K, Angew. Chem., Int. Ed 2021, 60, 41 22359. [DOI] [PMC free article] [PubMed] [Google Scholar]
[114].Zhao T, Beckwith JS, Amin MJ, Pálmai M, Snee PT, Tien M, Yang H, The J. Chem. Phys 2021, 155, 16 164201. [DOI] [PubMed] [Google Scholar]
[115].Keller AM, Devore MS, Stich DG, Vu DM, Causgrove T, Werner JH, Anal. Chem 2018, 90, 10 6109. [DOI] [PubMed] [Google Scholar]
[116].McHale K, Mabuchi H, Biophys. J 2010, 99, 1 313. [DOI] [PMC free article] [PubMed] [Google Scholar]
[117].Wells NP, Lessard GA, Goodwin PM, Phipps ME, Cutler PJ, Lidke DS, Wilson BS, Werner JH, Nano Lett. 2010, 10, 11 4732. [DOI] [PMC free article] [PubMed] [Google Scholar]
[118].Bopp MA, Sytnik A, Howard TD, Cogdell RJ, Hochstrasser RM, Proc. Natl. Acad. Sci. U. S. A 1999, 96, 20 11271. [DOI] [PMC free article] [PubMed] [Google Scholar]
[119].Schlau-Cohen GS, Wang Q, Southall J, Cogdell RJ, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2013, 110, 27 10899. [DOI] [PMC free article] [PubMed] [Google Scholar]
[120].Saga Y, Wazawa T, Mizoguchi T, Ishii Y, Yanagida T, Tamiaki H, Photochem. Photobiol 2002, 75, 4 433. [DOI] [PubMed] [Google Scholar]
[121].Ying L, Xie XS, J. Phys. Chem. B 1998, 102, 50 10399. [Google Scholar]
[122].Wang Q, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2015, 112, 45 13880. [DOI] [PMC free article] [PubMed] [Google Scholar]
[123].Wahadoszamen M, Krüger TP, Ara AM, van Grondelle R, Gwizdala M, Biochim. Biophys. Acta, Bioenerg 2020, 1861, 7. [DOI] [PubMed] [Google Scholar]
[124].Krüger TP, Novoderezhkin VI, Ilioaia C, Van Grondelle R, Biophys. J 2010, 98, 12 3093. [DOI] [PMC free article] [PubMed] [Google Scholar]
[125].Krüger TPJ, Wientjes E, Croce R, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 33 13516. [DOI] [PMC free article] [PubMed] [Google Scholar]
[126].Krüger TP, Malý P, Alexandre MT, Mančal T, Büchel C, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2017, 114, 52 E11063. [DOI] [PMC free article] [PubMed] [Google Scholar]
[127].Gwizdala M, Krüger TP, Wahadoszamen M, Gruber JM, Van Grondelle R, J. Phys. Chem. Lett 2018, 9, 6 1365. [DOI] [PubMed] [Google Scholar]
[128].Gwizdala M, Botha JL, Wilson A, Kirilovsky D, Van Grondelle R, Krüger TP, J. Phys. Chem. Lett 2018, 9, 9 2426. [DOI] [PubMed] [Google Scholar]
[129].Wientjes E, Renger J, Curto AG, Cogdell R, Van Hulst NF, Nat. Commun 2014, 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
[130].Kyeyune F, Botha JL, Van Heerden B, Malý P, Van Grondelle R, Diale M, Krüger TP, Nanoscale 2019, 11, 32 15139. [DOI] [PubMed] [Google Scholar]
[131].Liu C, Liu YL, Perillo EP, Jiang N, Dunn AK, Yeh HC, Appl. Phys. Lett 2015, 107, 15 153701. [DOI] [PMC free article] [PubMed] [Google Scholar]
[132].Kis-Petikova K, Gratton E, Microsc. Res. Tech 2004, 63, 1 34. [DOI] [PubMed] [Google Scholar]
[133].Åström KJ, Hägglund T, Advanced PID control, volume 461, The International Society of Automation, 2006. [Google Scholar]
[134].O’Dwyer A, Handbook of PI and PID controller tuning rules, World Scientific, 2009. [Google Scholar]
[135].Franklin GF, Powell JD, Emami-Naeini A, Feedback Control of Dynamic Systems, Prentice Hall, 2019. [Google Scholar]
[136].Berglund AJ, Mabuchi H, Appl. Phys. B: Lasers Opt 2004, 78, 5 653. [Google Scholar]
[137].Ha T, Chemla DS, Enderle T, Weiss S, Appl. Phys. Lett 1997, 70, 6 782. [Google Scholar]
[138].Andersson SB, In Proceedings of the 2010 American Control Conference, ACC 2010. IEEE, 2010. 4981–4986. [Google Scholar]
[139].Ashley TT, Chan-Tse C, Andersson SB, In Proceedings of the IEEE Conference on Decision and Control. IEEE, 2012. 2328–2333. [Google Scholar]
[140].Ashley TT, Andersson SB, Proc. Am. Control Conf. 2016, 2016-July, 2 6103. [Google Scholar]
[141].Ashley TT, Andersson SB, IEEE Trans. Autom. Control 2018, 63, 9 2855. [Google Scholar]
[142].Pinto SC, Vickers NA, Sharifi F, Andersson SB, In Proc. Am. Control Conf., volume 2021-May. 2021. 4033–4038. [DOI] [PMC free article] [PubMed] [Google Scholar]
[143].Pinto SC, Andersson SB, In 2021 60th IEEE Conference on Decision and Control (CDC). IEEE, 2021. 679–684. [DOI] [PMC free article] [PubMed] [Google Scholar]
[144].Ram S, Ward ES, Ober RJ, Multidimens. Syst. Signal Process 2006, 17, 1 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
[145].Gallatin GM, Berglund AJ, Opt. Express 2012, 20, 15 16381. [Google Scholar]
[146].Chao J, Ward ES, Ober RJ, J. Opt. Soc. Am. A 2016, 33, 7 B36. [DOI] [PMC free article] [PubMed] [Google Scholar]
[147].Mertz J, Introduction to optical microscopy, Cambridge University Press, 2019. [Google Scholar]
[148].Berglund AJ, Phys. Rev. E 2010, 82, 1 11917. [DOI] [PubMed] [Google Scholar]
[149].Calderon CP, Phys. Rev. E 2016, 93, 5 053303. [DOI] [PubMed] [Google Scholar]
[150].Juette MF, Gould TJ, Lessard MD, Mlodzianoski MJ, Nagpure BS, Bennett BT, Hess ST, Bewersdorf J, Nat. Methods 2008, 5, 6 527. [DOI] [PubMed] [Google Scholar]
[151].Juette MF, Rivera-Molina FE, Toomre DK, Bewersdorf J, Appl. Phys. Lett 2013, 102, 17 173702. [Google Scholar]
[152].Liu SL, Wang ZG, Xie HY, Liu AA, Lamb DC, Pang DW, Chem. Rev 2020, 120, 3 1936. [DOI] [PMC free article] [PubMed] [Google Scholar]
[153].Vahid MR, Hanzon B, Ober RJ, SIAM J. Imaging Sci 2020, 13, 1 234. [Google Scholar]
[154].d’Avigneau AM, Singh SS, Ober RJ, SIAM J. Imaging Sci 2022, 15, 1 139. [Google Scholar]
[155].Ashley TT, Andersson SB, Phys. Rev. E: Stat., Nonlinear, Soft Matter Phys 2015, 92, 5 1. [DOI] [PubMed] [Google Scholar]
[156].Godoy BI, Vickers NA, Andersson SB, Molecules 2021, 26, 4 886. [DOI] [PMC free article] [PubMed] [Google Scholar]
[157].Watkins LP, Yang H, Biophys. J 2004, 86 4015. [DOI] [PMC free article] [PubMed] [Google Scholar]
[158].Watkins LP, Yang H, The J. of Phys. Chem. B 2005. [DOI] [PubMed] [Google Scholar]
[159].Fields AP, Cohen AE, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 22 8937. [DOI] [PMC free article] [PubMed] [Google Scholar]
[160].Andersson SB, In 2007 46th IEEE Conference on Decision and Control. IEEE, 2007. 4950–4955. [Google Scholar]
[161].Bewersdorf J, Juette MF, Biophys. J 2011, 100, 3 617a. [Google Scholar]
[162].Fields AP, Cohen AE, Opt. Express 2012, 20, 20 22585. [DOI] [PubMed] [Google Scholar]
[163].Ernst D, Hain S, Köhler J, J. Opt. Soc. Am. A 2012, 29, 7 1277. [DOI] [PubMed] [Google Scholar]
[164].Casella G, Berger RL, Statistical inference, Cengage Learning, 2021. [Google Scholar]
[165].Zacks S, The theory of statistical inference., Wiley series in probability and mathematical statistics. Wiley, New York, 1971. [Google Scholar]
[166].Zhou Y, Handley M, Carles G, Harvey AR, APL Photonics 2019, 4, 6 060901. [Google Scholar]
[167].Shechtman Y, Sahl SJ, Backer AS, Moerner WE, Phys. Rev. Lett 2014, 113, 13 133902. [DOI] [PMC free article] [PubMed] [Google Scholar]
[168].Berglund AJ, Mabuchi H, Appl. Phys. B: Lasers Opt 2006, 83, 1 127. [Google Scholar]
[169].Juskatis R, In Pawley J, editor, Handbook of Biological Confocal Microscopy, 239–250. Springer, Boston, 2006. [Google Scholar]
[170].Petrov PN, Moerner W, Opt. Express 2020, 28, 13 18616. [DOI] [PMC free article] [PubMed] [Google Scholar]
[171].Lien C-H, Lin C-Y, Chen S-J, Chien F-C, Cheng C, Chang C.-y., Lin C.-y., Cho K.-c., Yen W.-c., Chang N.-s., Xu C, Dong CY, Chen S.-j., Yih JN, Hu YY, Sie YD, Cheng L.-c., Lien CH, Lien C.-h., Da Sie Y, Chien F.-c., Kim a., Miko S, Perret E, Olivo-Marin J.-c., Shorte S, Charneau P, Opt. Express 2014, 22, 22 27290. [DOI] [PubMed] [Google Scholar]
[172].Ernst D, Köhler J, Phys. Chem. Chem. Phys 2013, 15, 3 845. [DOI] [PubMed] [Google Scholar]
[173].Michalet X, Berglund AJ, Phys. Rev. E 2012, 85, 6 061916. [DOI] [PMC free article] [PubMed] [Google Scholar]
[174].Saxton MJ, Biophys. J 2012, 103, 12 2411. [DOI] [PMC free article] [PubMed] [Google Scholar]
[175].Du K, Liddle JA, Berglund AJ, Langmuir 2012, 28, 25 9181. [DOI] [PubMed] [Google Scholar]
[176].Vickers NA, Andersson SB, IFAC-PapersOnLine 2020, 53 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[177].Vickers NA, Andersson SB, IEEE Trans. Control Syst. Technol 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
[178].Coelho S, Baek J, Walsh J, Gooding JJ, Gaus K, Nat. Protoc 2021, 16, 1 497. [DOI] [PubMed] [Google Scholar]
[179].McGorty R, Kamiyama D, Huang B, Opt. Nanoscopy 2013, 2, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
[180].Gaigalas AK, Li L, Henderson O, Vogt R, Barr J, Marti G, Weaver J, Schwartz A, J. Res. Natl. Inst. Stand. Technol 2001, 106, 2 381. [DOI] [PMC free article] [PubMed] [Google Scholar]
[181].Andersson SB, Sun T, Appl. Phys. B: Lasers Opt 2009, 94, 3 403. [Google Scholar]
[182].Carr AR, Ponjavic A, Basu S, McColl J, Santos AM, Davis S, Laue ED, Klenerman D, Lee SF, Biophys. J 2017, 112, 7 1444. [DOI] [PMC free article] [PubMed] [Google Scholar]
[183].Berglund AJ, McHale K, Mabuchi H, Opt. Lett 2007, 32, 2 145. [DOI] [PubMed] [Google Scholar]
[184].Berglund AJ, McHale K, Mabuchi H, Opt. Express 2007, 15, 12 7752. [DOI] [PubMed] [Google Scholar]
[185].Berglund AJ, Ph.d. thesis, California Institute of Technology, 2007. [Google Scholar]
[186].Johnson C, Exell J, Kuo J, Welsher K, Opt. Express 2019, 27, 25 36241. [DOI] [PMC free article] [PubMed] [Google Scholar]
[187].Taylor RW, Sandoghdar V, Nano Lett. 2019, 19, 8 4827. [DOI] [PMC free article] [PubMed] [Google Scholar]
[188].Taylor RW, Sandoghdar V, In Label-Free Super-Resolution Microscopy, 25–65. Springer, 2019. [Google Scholar]
[189].Squires AH, Lavania AA, Dahlberg PD, Moerner WE, Nano Lett. 2019, 19, 6 4112. [DOI] [PMC free article] [PubMed] [Google Scholar]
[190].Mandel L, Wolf E, Optical Coherence and Quantum Optics, Cambridge University Press, 1995. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supinfo

NIHMS1819935-supplement-supinfo.pdf^{(280.4KB, pdf)}

[R1] [1].Dupont A, Lamb DC, Nanoscale 2011, 3, 11 4532. [DOI] [PubMed] [Google Scholar]

[R2] [2].Chenouard N, Smal I, De Chaumont F, Maška M, Sbalzarini IF, Gong Y, Cardinale J, Carthel C, Coraluppi S, Winter M, Cohen AR, Godinez WJ, Rohr K, Kalaidzidis Y, Liang L, Duncan J, Shen H, Xu Y, Magnusson KE, Jaldén J, Blau HM, Paul-Gilloteaux P, Roudot P, Kervrann C, Waharte F, Tinevez JY, Shorte SL, Willemse J, Celler K, Van Wezel GP, Dan HW, Tsai YS, De Solórzano CO, Olivo-Marin JC, Meijering E, Nat. Methods 2014, 11, 3 281. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R3] [3].Manzo C, Garcia-Parajo MF, Rep. Prog. Phys 2015, 78, 12. [DOI] [PubMed] [Google Scholar]

[R4] [4].Liu Z, Lavis LD, Betzig E, Mol. Cell 2015, 58 644. [DOI] [PubMed] [Google Scholar]

[R5] [5].Mathai PP, Liddle JA, Stavis SM, Appl. Phys. Rev 2016, 3, 1 011105. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R6] [6].Shen H, Tauzin LJ, Baiyasi R, Wang W, Moringo N, Shuang B, Landes CF, Chem. Rev 2017, 117, 11 7331. [DOI] [PubMed] [Google Scholar]

[R7] [7].Von Diezmann A, Shechtman Y, Moerner WE, Chem. Rev 2017, 117, 11 7244. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R8] [8].Zhong Y, Wang G, Annu. Rev. Anal. Chem 2020, 18 50. [DOI] [PubMed] [Google Scholar]

[R9] [9].Thompson MA, Lew MD, Badieirostami M, Moerner WE, Nano Lett. 2010, 10, 1 211. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R10] [10].Zhong Y, Zhao L, Tyrlik PM, Wang G, J. Phys. Chem. C 2017, 121, 14 8023. [Google Scholar]

[R11] [11].Wang D, Wu H, Schwartz DK, Phys. Rev. Lett 2017, 119, 26. [DOI] [PubMed] [Google Scholar]

[R12] [12].Bouchard MB, Voleti V, Mendes CS, Lacefield C, Grueber WB, Mann RS, Bruno RM, Hillman EM, Nat. Photonics 2015, 9, 2 113. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] [13].Moerner WE, J. Phys. Chem. B 2002, 106, 5 910. [Google Scholar]

[R14] [14].Cang H, Shan Xu C, Yang H, Chem. Phys. Lett 2008, 457, 4–6 285. [Google Scholar]

[R15] [15].Welsher K, Yang H, Faraday Discuss. 2015, 184 359. [DOI] [PubMed] [Google Scholar]

[R16] [16].Hou S, Johnson C, Welsher K, Molecules 2019, 24, 15 2826. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R17] [17].Wehnekamp F, Plucińska G, Thong R, Misgeld T, Lamb DC, eLife 2019, 8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R18] [18].Levi V, Ruan QQ, Gratton E, Biophys. J 2005, 88, 4 2919. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] [19].Annibale P, Dvornikov A, Gratton E, Biomed. Opt. Express 2015, 6, 6 2181. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] [20].Hou S, Lang X, Welsher K, Opt. Lett 2017, 42, 12 2390. [DOI] [PubMed] [Google Scholar]

[R21] [21].McHale K, Berglund AJ, Mabuchi H, Nano Lett. 2007, 7, 11 3535. [DOI] [PubMed] [Google Scholar]

[R22] [22].Andersson SB, Appl. Phys. B: Lasers Opt 2005, 80, 7 809. [Google Scholar]

[R23] [23].Lanzanò L, Gratton E, Methods Appl. Fluoresc 2014, 2, 2 024010. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R24] [24].Cang H, Wong CM, Xu S, Rizvi AH, Yang H, Appl. Phys. Lett 2006, 88 223901. [Google Scholar]

[R25] [25].Enderlein J, Appl. Phys. B: Lasers Opt 2000, 71, 5 773. [Google Scholar]

[R26] [26].Berglund AJ, Mabuchi H, Opt. Express 2005, 13, 20 8069. [DOI] [PubMed] [Google Scholar]

[R27] [27].Cohen AE, Moerner WE, Opt. Express 2008, 16, 10 6941. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R28] [28].Levi V, Ruan Q, Kis-Petikova K, Gratton E, Biochem. Soc. Trans 2003, 31, 5 997. [DOI] [PubMed] [Google Scholar]

[R29] [29].Wang Q, Moerner WE, Appl. Phys. B: Lasers Opt 2010, 99, 1–2 23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R30] [30].Wang Q, Moerner WE, ACS Nano 2011, 5, 7 5792. [DOI] [PubMed] [Google Scholar]

[R31] [31].van Heerden B, Krüger TPJ, In Gregor I, Erdmann R, Koberling F, editors, Single Molecule Spectroscopy and Superresolution Imaging XIV. SPIE, 2021. 22. [Google Scholar]

[R32] [32].Shen Z, Andersson SB, Proc. Am. Control Conf 2009, 1682–1687. [Google Scholar]

[R33] [33].Shen Z, Andersson SB, Proceedings of the IEEE Conference on Decision and Control 2009, 6052–6057. [Google Scholar]

[R34] [34].Shen Z, Andersson SB, Proceedings of the IEEE Conference on Decision and Control 2011, 5856–5861. [Google Scholar]

[R35] [35].Shen Z, Andersson SB, IEEE Trans. Control Syst. Technol 2011, 19, 5 1269. [Google Scholar]

[R36] [36].Shen Z, Andersson SB, Mechatronics 2012, 22, 3 320. [Google Scholar]

[R37] [37].Zhang C, Welsher K, Entropy 2021, 23, 5 498. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] [38].Balzarotti F, Eilers Y, Gwosch KC, Gynnå AH, Westphal V, Stefani FD, Elf J, Hell SW, Science 2017, 355, 6325 606. [DOI] [PubMed] [Google Scholar]

[R39] [39].Gwosch KC, Pape JK, Balzarotti F, Hoess P, Ellenberg J, Ries J, Hell SW, Nat. Methods 2020, 17, 2 217. [DOI] [PubMed] [Google Scholar]

[R40] [40].Müller BK, Zaychikov E, Bräuchle C, Lamb DC, Biophys. J 2005, 89, 5 3508. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R41] [41].Dertinger T, Koberling F, Benda A, Erdmann R, Hof M, Enderlein J, In Nicolau DV, Enderlein J, Leif RC, Farkas DL, Raghavachari R, editors, Imaging, Manipulation, and Analysis of Biomolecules and Cells: Fundamentals and Applications III, volume 5699. SPIE, 2005. 219. [Google Scholar]

[R42] [42].Davis L, Sikorski Z, Robinson W, Shen G, Li X, Canfield B, Lescano I, Bomar B, Hofmeister W, Germann J, King J, White Y, Terekhov A, In Enderlein J, Gryczynski ZK, Erdmann R, editors, Single Molecule Spectroscopy and Imaging, volume 6862. SPIE, 2008. 68620P. [Google Scholar]

[R43] [43].Gremann JA, Davis LM, Opt. Express 2014, 22, 5 5641. [DOI] [PubMed] [Google Scholar]

[R44] [44].Perillo EP, Liu YL, Huynh K, Liu C, Chou CK, Hung MC, Yeh HC, Dunn AK, Nat. Commun 2015, 6, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R45] [45].Liu YL, Perillo EP, Ang P, Kim M, Nguyen DT, Blocher K, Chen YA, Liu C, Hassan AM, Vu HT, Chen YI, Dunn AK, Yeh HC, ACS Nano 2020, 14, 7 7927. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R46] [46].Masullo LA, Steiner F, Zähringer J, Lopez LF, Bohlen J, Richter L, Cole F, Tinnefeld P, Stefani FD, Nano Lett. 2021, 21, 1 840. [DOI] [PubMed] [Google Scholar]

[R47] [47].Dissanayaka KD, Canfield BK, Davis LM, Opt. Express 2019, 27, 21 29759. [DOI] [PubMed] [Google Scholar]

[R48] [48].Andersson SB, Appl. Phys. B: Lasers Opt 2011, 104, 1 161. [Google Scholar]

[R49] [49].Ashley TT, Gan EL, Pan J, Andersson SB, Biomed. Opt. Express 2016, 7, 9 3355. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R50] [50].Vickers NA, Andersson SB, IFAC-PapersOnLine 2021, 54, 7 649. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R51] [51].Cang H, Xu CS, Montiel D, Yang H, Xu S, Montiel D, Yang H, Opt. Lett 2007, 32, 18 2729. [DOI] [PubMed] [Google Scholar]

[R52] [52].Lessard GA, Goodwin PM, Werner JH, In Ultrasensitive and Single-Molecule Detection Technologies, volume 6092. SPIE, 2006. 609205. [Google Scholar]

[R53] [53].Lessard GA, Goodwin PM, Werner JH, Appl. Phys. Lett 2007, 91, 22 224106. [Google Scholar]

[R54] [54].DeVore MS, Stich DG, Keller AM, Ghosh Y, Goodwin PM, Phipps ME, Stewart MH, Cleyrat C, Wilson BS, Lidke DS, Hollingsworth JA, Werner JH, In Parak WJ, Osinski M, Liang X-J, editors, Colloidal Nanoparticles for Biomedical Applications X, volume 9338. SPIE, 2015. 933812. [Google Scholar]

[R55] [55].Dunlap MK, Ryan DP, Goodwin PM, Werner JH, Majumder S, Hollingsworth JA, Gelfand MP, Van Orden A, Appl. Phys. Lett 2020, 116, 2 21103. [Google Scholar]

[R56] [56].Juette MF, Bewersdorf J, Nano Lett. 2010, 10, 11 4657. [DOI] [PubMed] [Google Scholar]

[R57] [57].Spille JH, Kaminski TP, Scherer K, Rinne JS, Heckel A, Kubitscheck U, Nucleic Acids Res. 2015, 43, 2 e14. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R58] [58].Wang Q, Goldsmith RH, Jiang Y, Bockenhauer SD, Moerner WE, Acc. Chem. Res 2012, 45, 11 1955. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R59] [59].Bespalova MI, Mahanta S, Krishnan M, Curr. Opin. Chem. Biol 2019, 51 113. [DOI] [PubMed] [Google Scholar]

[R60] [60].Sahl SJ, Leutenegger M, Hilbert M, Hell SW, Eggeling C, Proc. Natl. Acad. Sci. U. S. A 2010, 107, 15 6829. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R61] [61].Sahl SJ, Leutenegger M, Hell SW, Eggeling C, ChemPhysChem 2014, 15, 4 771. [DOI] [PubMed] [Google Scholar]

[R62] [62].Jazani S, Sgouralis I, Pressé S, J. Chem. Phys 2019, 150, 11 114108. [DOI] [PubMed] [Google Scholar]

[R63] [63].So PTC, Ragan T, Gratton E, Carerro J, Voss E, In Farkas DL, Tromberg BJ, editors, Functional Imaging and Optical Manipulation of Living Cells, volume 2983. SPIE, 1997. 45–56. [Google Scholar]

[R64] [64].Peters IM, De Grooth BG, Schins JM, Figdor CG, Greve J, Rev. Sci. Instrum 1998, 69, 7 2762. [Google Scholar]

[R65] [65].Ragan T, Huang H, So P, Gratton E, J. Fluoresc 2006, 16, 3 325. [DOI] [PubMed] [Google Scholar]

[R66] [66].Hellriegel C, Gratton E, J. R. Soc., Interface 2009, 6, SUPPL. 1 S3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R67] [67].Katayama Y, Burkacky O, Meyer M, Bräuchle C, Gratton E, Lamb DC, ChemPhysChem 2009, 10, 14 2458. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R68] [68].Reuel NF, Dupont A, Thouvenin O, Lamb DC, Strano MS, ACS Nano 2012, 6, 6 5420. [DOI] [PubMed] [Google Scholar]

[R69] [69].Dupont A, Gorelashvili M, Schüller V, Wehnekamp F, Arcizet D, Katayama Y, Lamb DC, Heinrich D, New J. Phys 2013, 15, 7 075008. [Google Scholar]

[R70] [70].Verdeny-Vilanova I, Wehnekamp F, Mohan N, Álvarez á. S., Borbely JS, Otterstrom JJ, Lamb DC, Lakadamyali M, J. Cell Sci 2017, 130, 11 1904. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R71] [71].Donovan BT, Huynh A, Ball DA, Patel HP, Poirier MG, Larson DR, Ferguson ML, Lenstra TL, The EMBO Journal 2019, 38, 12 e100809. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R72] [72].Marklund E, van Oosten B, Mao G, Amselem E, Kipper K, Sabantsev A, Emmerich A, Globisch D, Zheng X, Lehmann LC, et al. , Nature 2020, 583, 7818 858. [DOI] [PubMed] [Google Scholar]

[R73] [73].Lanzano L, Digman MA, Fwu P, Giral H, Levi M, Gratton E, Biophotonics J 2011, 4, 6 415. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R74] [74].Mieskes F, Wehnekamp F, Plucińska G, Thong R, Misgeld T, Lamb DC, Data in Brief 2020, 29 105280. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R75] [75].Hou S, Welsher K, Small 2019, 15, 44. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R76] [76].Patel S, Ashwanikumar N, Robinson E, Xia Y, Mihai C, Griffith JP, Hou S, Esposito AA, Ketova T, Welsher K, Joyal JL, Almarsson Ö, Sahay G, Nat. Commun 2020, 11, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R77] [77].Schmidt R, Weihs T, Wurm CA, Jansen I, Rehman J, Sahl SJ, Hell SW, Nat. Commun 2021, 12, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R78] [78].Pape JK, Stephan T, Balzarotti F, Büchner R, Lange F, Riedel D, Jakobs S, Hell SW, Proc. Natl. Acad. Sci. U. S. A 2020, 117, 34 20607. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R79] [79].Liu YL, Perillo EP, Liu C, Yu P, Chou CK, Hung MC, Dunn AK, Yeh HC, Biophys. J 2016, 111, 10 2214. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R80] [80].Wells NP, Lessard GA, Phipps ME, Goodwin PM, Lidke DS, Wilson BS, Werner JH, In Single Molecule Spectroscopy and Imaging II, volume 7185. SPIE, 2009. 71850Z. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R81] [81].DeVore MS, Stich DG, Keller AM, Cleyrat C, Phipps ME, Hollingsworth JA, Lidke DS, Wilson BS, Goodwin PM, Werner JH, Rev. Sci. Instrum 2015, 86, 12. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R82] [82].Welsher K, Yang H, Nat. Nanotechnol 2014, 9, 3 198. [DOI] [PubMed] [Google Scholar]

[R83] [83].Ha T, Biochemistry 2004, 43, 14 4055. [DOI] [PubMed] [Google Scholar]

[R84] [84].Alhadid Y, Chung SY, Lerner E, Taatjes DJ, Borukhov S, Weiss S, Protein Sci. 2017, 26, 7 1278. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R85] [85].Kaur G, Lewis JS, Van Oijen AM, Molecules 2019, 24, 3 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R86] [86].English BP, Min W, Van Oijen AM, Kang TL, Luo G, Sun H, Cherayil BJ, Kou SC, Xie XS, Nat. Chem. Biol 2006, 2, 2 87. [DOI] [PubMed] [Google Scholar]

[R87] [87].Jiang Y, Douglas NR, Conley NR, Miller EJ, Frydman J, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 41 16962. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R88] [88].Yildiz A, Tomishige M, Vale RD, Selvin PR, Science 2004, 303, 5658 676. [DOI] [PubMed] [Google Scholar]

[R89] [89].Dienerowitz M, Howard JA, Quinn SD, Dienerowitz F, Leake MC, Methods 2021, 193 96. [DOI] [PubMed] [Google Scholar]

[R90] [90].Krüger TP, Van Grondelle R, Physica B: Condense. Matter 2016, 480 7. [Google Scholar]

[R91] [91].Kondo T, Chen WJ, Schlau-Cohen GS, Chem. Rev 2017, 117, 2 860. [DOI] [PubMed] [Google Scholar]

[R92] [92].Gruber JM, Malý P, Krüger TPJ, Grondelle RV, Nanophotonics 2018, 7, 1 81. [Google Scholar]

[R93] [93].Mazal H, Haran G, Curr. Opin. Biomed. Eng 2019, 12 8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R94] [94].Kapanidis AN, Margeat E, Ho SO, Kortkhonjia E, Weiss S, Ebright RH, Science 2006, 314, 5802 1144. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R95] [95].Hou S, Exell J, Welsher K, Nat. Commun 2020, 11, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R96] [96].Ha T, Ting AY, Liang J, Caldwell WB, Deniz AA, Chemla DS, Schultz PG, Weiss S, Proc. Natl. Acad. Sci. U. S. A 1999, 96, 3 893. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R97] [97].Krüger TP, Ilioaia C, Johnson MP, Ruban AV, Papagiannakis E, Horton P, Van Grondelle R, Biophys. J 2012, 102, 11 2669. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R98] [98].Schlau-Cohen GS, Yang HY, Krüger TPJ, Xu P, Gwizdala M, Van Grondelle R, Croce R, Moerner WE, J. Phys. Chem. Lett 2015, 6, 5 860. [DOI] [PubMed] [Google Scholar]

[R99] [99].Gwizdala M, Berera R, Kirilovsky D, Van Grondelle R, Krüger TP, J. Am. Chem. Soc 2016, 138, 36 11616. [DOI] [PubMed] [Google Scholar]

[R100] [100].Thyrhaug E, Krause S, Perri A, Cerullo G, Polli D, Vosch T, Hauer J, Proc. Natl. Acad. Sci. U. S. A 2019, 116, 10 4064. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R101] [101].Van Dijk EM, Hernando J, García-Parajó MF, Van Hulst NF, J. Chem. Phys 2005, 123, 6. [DOI] [PubMed] [Google Scholar]

[R102] [102].Hildner R, Brinks D, Nieder JB, Cogdell RJ, Van Hulst NF, Science 2013, 340, 6139 1448. [DOI] [PubMed] [Google Scholar]

[R103] [103].Malý P, Gruber JM, Cogdell RJ, Mančal T, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2016, 113, 11 2934. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R104] [104].Liebel M, Toninelli C, Van Hulst NF, Nat. Photonics 2018, 12, 1 45. [Google Scholar]

[R105] [105].Moya R, Kondo T, Norris AC, Schlau-Cohen GS, Opt. Express 2021, 29, 18 28246. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R106] [106].Moya R, Norris AC, Kondo T, Schlau-Cohen GS, Nat. Chem 2022, 14, 2153. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R107] [107].Haran G, Acc. Chem. Res 2010, 43, 8 1135. [DOI] [PubMed] [Google Scholar]

[R108] [108].McHale K, Mabuchi H, J. Am. Chem. Soc 2009, 131, 49 17901. [DOI] [PubMed] [Google Scholar]

[R109] [109].Wells NP, Lessard GA, Werner JH, Anal. Chem 2008, 80, 24 9830. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R110] [110].Han JJ, Kiss C, Bradbury AR, Werner JH, ACS Nano 2012, 6, 10 8922. [DOI] [PubMed] [Google Scholar]

[R111] [111].Liu C, Obliosca JM, Liu YL, Chen YA, Jiang N, Yeh HC, Nanoscale 2017, 9, 17 5664. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R112] [112].Chen Y-I, Chang Y-J, Nguyen TD, Liu C, Phillion S, Kuo Y-A, Vu HT, Liu A, Liu Y-L, Hong S, Ren P, Yankeelov TE, Yeh H-C, J. Am. Chem. Soc 2019, 141, 40 15747. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R113] [113].Tan X, Welsher K, Angew. Chem., Int. Ed 2021, 60, 41 22359. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R114] [114].Zhao T, Beckwith JS, Amin MJ, Pálmai M, Snee PT, Tien M, Yang H, The J. Chem. Phys 2021, 155, 16 164201. [DOI] [PubMed] [Google Scholar]

[R115] [115].Keller AM, Devore MS, Stich DG, Vu DM, Causgrove T, Werner JH, Anal. Chem 2018, 90, 10 6109. [DOI] [PubMed] [Google Scholar]

[R116] [116].McHale K, Mabuchi H, Biophys. J 2010, 99, 1 313. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R117] [117].Wells NP, Lessard GA, Goodwin PM, Phipps ME, Cutler PJ, Lidke DS, Wilson BS, Werner JH, Nano Lett. 2010, 10, 11 4732. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R118] [118].Bopp MA, Sytnik A, Howard TD, Cogdell RJ, Hochstrasser RM, Proc. Natl. Acad. Sci. U. S. A 1999, 96, 20 11271. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R119] [119].Schlau-Cohen GS, Wang Q, Southall J, Cogdell RJ, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2013, 110, 27 10899. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R120] [120].Saga Y, Wazawa T, Mizoguchi T, Ishii Y, Yanagida T, Tamiaki H, Photochem. Photobiol 2002, 75, 4 433. [DOI] [PubMed] [Google Scholar]

[R121] [121].Ying L, Xie XS, J. Phys. Chem. B 1998, 102, 50 10399. [Google Scholar]

[R122] [122].Wang Q, Moerner WE, Proc. Natl. Acad. Sci. U. S. A 2015, 112, 45 13880. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R123] [123].Wahadoszamen M, Krüger TP, Ara AM, van Grondelle R, Gwizdala M, Biochim. Biophys. Acta, Bioenerg 2020, 1861, 7. [DOI] [PubMed] [Google Scholar]

[R124] [124].Krüger TP, Novoderezhkin VI, Ilioaia C, Van Grondelle R, Biophys. J 2010, 98, 12 3093. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R125] [125].Krüger TPJ, Wientjes E, Croce R, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 33 13516. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R126] [126].Krüger TP, Malý P, Alexandre MT, Mančal T, Büchel C, Van Grondelle R, Proc. Natl. Acad. Sci. U. S. A 2017, 114, 52 E11063. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R127] [127].Gwizdala M, Krüger TP, Wahadoszamen M, Gruber JM, Van Grondelle R, J. Phys. Chem. Lett 2018, 9, 6 1365. [DOI] [PubMed] [Google Scholar]

[R128] [128].Gwizdala M, Botha JL, Wilson A, Kirilovsky D, Van Grondelle R, Krüger TP, J. Phys. Chem. Lett 2018, 9, 9 2426. [DOI] [PubMed] [Google Scholar]

[R129] [129].Wientjes E, Renger J, Curto AG, Cogdell R, Van Hulst NF, Nat. Commun 2014, 5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R130] [130].Kyeyune F, Botha JL, Van Heerden B, Malý P, Van Grondelle R, Diale M, Krüger TP, Nanoscale 2019, 11, 32 15139. [DOI] [PubMed] [Google Scholar]

[R131] [131].Liu C, Liu YL, Perillo EP, Jiang N, Dunn AK, Yeh HC, Appl. Phys. Lett 2015, 107, 15 153701. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R132] [132].Kis-Petikova K, Gratton E, Microsc. Res. Tech 2004, 63, 1 34. [DOI] [PubMed] [Google Scholar]

[R133] [133].Åström KJ, Hägglund T, Advanced PID control, volume 461, The International Society of Automation, 2006. [Google Scholar]

[R134] [134].O’Dwyer A, Handbook of PI and PID controller tuning rules, World Scientific, 2009. [Google Scholar]

[R135] [135].Franklin GF, Powell JD, Emami-Naeini A, Feedback Control of Dynamic Systems, Prentice Hall, 2019. [Google Scholar]

[R136] [136].Berglund AJ, Mabuchi H, Appl. Phys. B: Lasers Opt 2004, 78, 5 653. [Google Scholar]

[R137] [137].Ha T, Chemla DS, Enderle T, Weiss S, Appl. Phys. Lett 1997, 70, 6 782. [Google Scholar]

[R138] [138].Andersson SB, In Proceedings of the 2010 American Control Conference, ACC 2010. IEEE, 2010. 4981–4986. [Google Scholar]

[R139] [139].Ashley TT, Chan-Tse C, Andersson SB, In Proceedings of the IEEE Conference on Decision and Control. IEEE, 2012. 2328–2333. [Google Scholar]

[R140] [140].Ashley TT, Andersson SB, Proc. Am. Control Conf. 2016, 2016-July, 2 6103. [Google Scholar]

[R141] [141].Ashley TT, Andersson SB, IEEE Trans. Autom. Control 2018, 63, 9 2855. [Google Scholar]

[R142] [142].Pinto SC, Vickers NA, Sharifi F, Andersson SB, In Proc. Am. Control Conf., volume 2021-May. 2021. 4033–4038. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R143] [143].Pinto SC, Andersson SB, In 2021 60th IEEE Conference on Decision and Control (CDC). IEEE, 2021. 679–684. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R144] [144].Ram S, Ward ES, Ober RJ, Multidimens. Syst. Signal Process 2006, 17, 1 27. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R145] [145].Gallatin GM, Berglund AJ, Opt. Express 2012, 20, 15 16381. [Google Scholar]

[R146] [146].Chao J, Ward ES, Ober RJ, J. Opt. Soc. Am. A 2016, 33, 7 B36. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R147] [147].Mertz J, Introduction to optical microscopy, Cambridge University Press, 2019. [Google Scholar]

[R148] [148].Berglund AJ, Phys. Rev. E 2010, 82, 1 11917. [DOI] [PubMed] [Google Scholar]

[R149] [149].Calderon CP, Phys. Rev. E 2016, 93, 5 053303. [DOI] [PubMed] [Google Scholar]

[R150] [150].Juette MF, Gould TJ, Lessard MD, Mlodzianoski MJ, Nagpure BS, Bennett BT, Hess ST, Bewersdorf J, Nat. Methods 2008, 5, 6 527. [DOI] [PubMed] [Google Scholar]

[R151] [151].Juette MF, Rivera-Molina FE, Toomre DK, Bewersdorf J, Appl. Phys. Lett 2013, 102, 17 173702. [Google Scholar]

[R152] [152].Liu SL, Wang ZG, Xie HY, Liu AA, Lamb DC, Pang DW, Chem. Rev 2020, 120, 3 1936. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R153] [153].Vahid MR, Hanzon B, Ober RJ, SIAM J. Imaging Sci 2020, 13, 1 234. [Google Scholar]

[R154] [154].d’Avigneau AM, Singh SS, Ober RJ, SIAM J. Imaging Sci 2022, 15, 1 139. [Google Scholar]

[R155] [155].Ashley TT, Andersson SB, Phys. Rev. E: Stat., Nonlinear, Soft Matter Phys 2015, 92, 5 1. [DOI] [PubMed] [Google Scholar]

[R156] [156].Godoy BI, Vickers NA, Andersson SB, Molecules 2021, 26, 4 886. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R157] [157].Watkins LP, Yang H, Biophys. J 2004, 86 4015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R158] [158].Watkins LP, Yang H, The J. of Phys. Chem. B 2005. [DOI] [PubMed] [Google Scholar]

[R159] [159].Fields AP, Cohen AE, Proc. Natl. Acad. Sci. U. S. A 2011, 108, 22 8937. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R160] [160].Andersson SB, In 2007 46th IEEE Conference on Decision and Control. IEEE, 2007. 4950–4955. [Google Scholar]

[R161] [161].Bewersdorf J, Juette MF, Biophys. J 2011, 100, 3 617a. [Google Scholar]

[R162] [162].Fields AP, Cohen AE, Opt. Express 2012, 20, 20 22585. [DOI] [PubMed] [Google Scholar]

[R163] [163].Ernst D, Hain S, Köhler J, J. Opt. Soc. Am. A 2012, 29, 7 1277. [DOI] [PubMed] [Google Scholar]

[R164] [164].Casella G, Berger RL, Statistical inference, Cengage Learning, 2021. [Google Scholar]

[R165] [165].Zacks S, The theory of statistical inference., Wiley series in probability and mathematical statistics. Wiley, New York, 1971. [Google Scholar]

[R166] [166].Zhou Y, Handley M, Carles G, Harvey AR, APL Photonics 2019, 4, 6 060901. [Google Scholar]

[R167] [167].Shechtman Y, Sahl SJ, Backer AS, Moerner WE, Phys. Rev. Lett 2014, 113, 13 133902. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R168] [168].Berglund AJ, Mabuchi H, Appl. Phys. B: Lasers Opt 2006, 83, 1 127. [Google Scholar]

[R169] [169].Juskatis R, In Pawley J, editor, Handbook of Biological Confocal Microscopy, 239–250. Springer, Boston, 2006. [Google Scholar]

[R170] [170].Petrov PN, Moerner W, Opt. Express 2020, 28, 13 18616. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R171] [171].Lien C-H, Lin C-Y, Chen S-J, Chien F-C, Cheng C, Chang C.-y., Lin C.-y., Cho K.-c., Yen W.-c., Chang N.-s., Xu C, Dong CY, Chen S.-j., Yih JN, Hu YY, Sie YD, Cheng L.-c., Lien CH, Lien C.-h., Da Sie Y, Chien F.-c., Kim a., Miko S, Perret E, Olivo-Marin J.-c., Shorte S, Charneau P, Opt. Express 2014, 22, 22 27290. [DOI] [PubMed] [Google Scholar]

[R172] [172].Ernst D, Köhler J, Phys. Chem. Chem. Phys 2013, 15, 3 845. [DOI] [PubMed] [Google Scholar]

[R173] [173].Michalet X, Berglund AJ, Phys. Rev. E 2012, 85, 6 061916. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R174] [174].Saxton MJ, Biophys. J 2012, 103, 12 2411. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R175] [175].Du K, Liddle JA, Berglund AJ, Langmuir 2012, 28, 25 9181. [DOI] [PubMed] [Google Scholar]

[R176] [176].Vickers NA, Andersson SB, IFAC-PapersOnLine 2020, 53 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R177] [177].Vickers NA, Andersson SB, IEEE Trans. Control Syst. Technol 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R178] [178].Coelho S, Baek J, Walsh J, Gooding JJ, Gaus K, Nat. Protoc 2021, 16, 1 497. [DOI] [PubMed] [Google Scholar]

[R179] [179].McGorty R, Kamiyama D, Huang B, Opt. Nanoscopy 2013, 2, 1 1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R180] [180].Gaigalas AK, Li L, Henderson O, Vogt R, Barr J, Marti G, Weaver J, Schwartz A, J. Res. Natl. Inst. Stand. Technol 2001, 106, 2 381. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R181] [181].Andersson SB, Sun T, Appl. Phys. B: Lasers Opt 2009, 94, 3 403. [Google Scholar]

[R182] [182].Carr AR, Ponjavic A, Basu S, McColl J, Santos AM, Davis S, Laue ED, Klenerman D, Lee SF, Biophys. J 2017, 112, 7 1444. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R183] [183].Berglund AJ, McHale K, Mabuchi H, Opt. Lett 2007, 32, 2 145. [DOI] [PubMed] [Google Scholar]

[R184] [184].Berglund AJ, McHale K, Mabuchi H, Opt. Express 2007, 15, 12 7752. [DOI] [PubMed] [Google Scholar]

[R185] [185].Berglund AJ, Ph.d. thesis, California Institute of Technology, 2007. [Google Scholar]

[R186] [186].Johnson C, Exell J, Kuo J, Welsher K, Opt. Express 2019, 27, 25 36241. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R187] [187].Taylor RW, Sandoghdar V, Nano Lett. 2019, 19, 8 4827. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R188] [188].Taylor RW, Sandoghdar V, In Label-Free Super-Resolution Microscopy, 25–65. Springer, 2019. [Google Scholar]

[R189] [189].Squires AH, Lavania AA, Dahlberg PD, Moerner WE, Nano Lett. 2019, 19, 6 4112. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R190] [190].Mandel L, Wolf E, Optical Coherence and Quantum Optics, Cambridge University Press, 1995. [Google Scholar]

PERMALINK

Real-Time Feedback-Driven Single-Particle Tracking: A Survey and Perspective

Bertus van Heerden

Nicholas A Vickers

Tjaart PJ Krüger

Sean B Andersson

Abstract

Graphical Abstract

1. Introduction

2. Survey

Figure 1:

2.1. Single-detector methods

Figure 2:

2.2. Multi-detector methods

2.3. Complementary methods

3. Applications

3.1. Microscopy and direct tracking-based applications

Figure 3:

3.2. Spectroscopy-based applications

Figure 4:

4. Feedback Tracking and Control

4.1. Controller Architecture

Figure 5:

4.2. Control Challenges and Limitations

Figure 6:

Photon collection:

Algorithm:

Actuation:

5. Estimation and Fisher Information

5.1. Estimators

5.2. Fisher Information and Cramér-Rao Lower Bounds

Figure 7:

6. Calibration and Characterization

Figure 8:

7. Tracking Performance

7.1. Current Performance

Figure 9:

Table 1:

7.2. Fundamental Performance and Trade-offs

7.3. Performance–Application Fit

8. Future Perspectives

8.1. Current and Future Trends

8.2. Open Challenges and Opportunities

8.3. Adoption of RT-FD-SPT

Supplementary Material

Acknowledgements

Biographies

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases