Abstract
Single particle tracking plays an important role in studying physical and kinetic properties of biomolecules. In this work, we introduce the application of Expectation Maximization (EM) based algorithms for solving localization and parameter estimation problems in SPT using data captured from scientific complementary metal-oxide semiconductor (sCMOS) camera sensors. Two representative methods are considered for generating the filtered and smoothed distributions needed by EM: Sequential Monte Carlo - EM, and Unscented - EM. The SMC method uses particle filtering and particle smoothing to handle general distributions, while the U scheme reduces the computational burden through the use of an unscented Kalman Filter and an unscented Rauch-Tung Striebel Smoother. We also investigate the influence of the number of images in the dataset on the final estimates through intensive simulations as well as the computational efficiency of the two methods.
I. Introduction
Single Particle Tracking (SPT) plays an important role in studying the physical properties and dynamics of biomolecules. The targets of interest, such as viruses or proteins, are nanometer-scale and not resolvable with standard optical microscopy. Their motion can be revealed, however, by labeling them with a fluorescent tag, such as a quantum dot or fluorescent protein, and imaging the resulting fluorescence signal. While SPT encompasses many experimental techniques [1]–[3], in general, measurements about the system come in the form of a sequence of images taken by a camera. These images are then analyzed to determine particle trajectories and physical and kinetic parameters.
To date, many algorithms have been developed for analyzing SPT datasets. Under the standard paradigm, a two-step process is applied in which images are first processed individually to determine the location of each particle in a frame and these positions linked across frames to form trajectories. In the second step, trajectories are analyzed to extract information about the dynamic process, such as the value of the diffusion coefficient or other motion parameters [4]–[6]. Though the performance of these methods is good when the signal level is high and the motion model simple, they begin to fail as the signal level decreases or model complexity increases.
We previously introduced an approach based on nonlinear system identification that uses Expectation Maximization (EM) combined with particle filtering and smoothing to analyze segemented image data (that is, each image sequence contained information about a single particle) [7]. (The segementation step, while not trivial, is a standard processing step in SPT algorithms.) This general approach, termed Sequential Monte Carlo-EM (SMC-EM), can handle nearly arbitrary nonlinearities in both the motion and observation models and has been shown to work at least as well as state-of-the-art methods for 2-D diffusion. However, this advantage comes at the cost of high computational complexity. This issue was then addressed by replacing the particle-based methods with an Unscented Kalman filter (UKF) and Unscented Rauch-Tung-Striebel smoother (URTSS), a scheme we refer to as Unscented-EM (U-EM) [8].
In this work, we build upon our EM-based algorithms, extending them to handle SPT data from cameras with pixel-dependent readout noise, an intrinsic characteristic of the scientific complementary metal-oxide semiconductor (sCMOS) camera. sCMOS cameras are increasingly popular due to their high frame rate, large imaging area, high sensitivity, and relatively low cost [9], [10]. The unique architecture of sCMOS camera sensors leads the readout noise to vary from pixel to pixel. Failing to account for the unique characteristics of readout noise in each pixel has a negative impact on the quality of estimation [9].
Combining the photon detection process of the microscope with the read-out noise of the detector leads to a nonlinear, non-Gaussian measurement model. While this can be handled directly using SMC methods, the UKF requires Gaussian distributed noise. We thereore apply a Generalized Anscombe Transformation to turn the measurement model into a form that is amenable to UKF. Finally, we combine this observation model with an Ornstein-Uhlenbeck motion model, a common model of motion for biomolecular processes that combines diffusion with a restorative force. We study the relative performance of SMC-EM and U-EM under this scenario. In addition, since the quality of the final estimates depends both on the chosen algorithm and the amount of available data, we also consider the impact of the number of camera frames available for analysis.
The remainder of the paper is organized as follows. In Sec. II, we present the motion and observation models. In Sec. III, we briely review our EM-based algorithms. In Sec. IV, we demonstrate the efficacy of the EM-based algorithms to SPT using sCMOS camera and investigate the influence of image length on final localization and parameter estimation performance. We also discuss the computation time for the different approaches. Finally, we make a few concluding remarks in Sec. V.
II. Problem Formulation
A. Motion model
For simplicity of presentation, and as is commonly assumed in the SPT literature, we take the motion in each axis to be independent. Consider, then, a generic linear motion model in a single axis given by
| (1) |
where denotes the position in one direction, denotes a constant “velocity” term, and denotes a Gaussian white noise stochastic process. The model (1) can describe a variety of models important to biomolecular motion, including pure diffusion, Ornstein-Uhlenbeck (O-U) processes, directed flow and combinations of these.
In this work, we focus on the O-U process. Motivated by the model presented in [11] where O-U is used to describe a molecule tethered to a surface by a flexible chain, we set a and Q in (1) as
| (2a) |
| (2b) |
where Δt′ is defined by the frame rate of the camera, A > 0 is the stiffness coefficient, and D is the diffusion coefficient. These important physical parameters are usually unknown and need to be estimated to reveal properrties of the biomolecular motion. Note that with appropriate definitions of the parameters (a, Q), the O-U model can approximate confined diffusion [12].)
The state transition probability density between two successive images under the O-U model is
| (3) |
B. Observation model
The output of the camera is a sequence of images that are usually segmented into small regions, each given by a pixelated square array and containing information about a single particle. The size of each pixel, determined by the physical size of the camera element and the optimal magnification, is denoted as Δx by Δy. At time step t, the intensity generated by a fluorescent particle is given by a Poisson random variable with a rate given by the expected photon intensity for the pth pixel,
| (4) |
where (xt, yt) is the position of the fluorescent particle, (, , ,) are the integration bounds over the boundaries of the given pixel, G denotes the peak intensity of the fluorescence, and PSF represents the point spread function of the instrument. For objects in the focal plane of the instrument, this function is well-approximated by
| (5) |
where the particle is located at the origin, the pair (x, y) represents a position on the plane x-y at which the PSF is being evaluated, and σx and σy are given by
| (6) |
Here λ is the wavelength of the emitted light and NA is the numerical aperture of the objective lens being used [13].
In addition to the Poisson nature of the signal, there is always additional noise arising from background fluorescence (also Poisson in nature) and from the read-out electronics of the camera. For the small images with P pixels, the background intensity rate can be taken to be a constant, Nbgd. For the read-out noise, we focus on an sCMOS camera sensor. These devices have pixel-dependent statistics, leading to the model
| (7a) |
| (7b) |
| (7c) |
where Poiss(·) represents a Poisson distribution, σp,t is the standard deviation of the readout noise, and Varp,t and gp,t are named variance and gain for the pth pixel at time t respectively.
C. Measurement model transformation
One of the algorithms described in Sec. III relies on a UKF. This filter is applicable to nonlinear observation models with additive Gaussian noise [14]. However, the observation model we considered in Sec.II-B is the convolution of a Poisson distribution and a Gaussian distribution. Therefore, we seek to transform the model in (7) into a form amenable to the UKF. There are different approaches for variance stabilization, such as the Anscombe [15] or the Freeman and Tukey [16] transformations, or direct approximation by a Gaussian model (when measured intensities are sufficiently large) [17]. These different approaches have been discussed and compared in [8] where it was found that in general the Anscombe transform performs the best in the SPT setting. Therefore, we use the generalized Anscombe transformation [18] to transform the observation model (7) into
| (8) |
To apply this transformation, the observed measurements Ip,t should be first expressed as
| (9) |
We then take the transformed observation as the input for the U-EM method described in Sec.III-C.
III. Inference Problem
Our general scheme for EM-based analysis of SPT data is shown in Fig. 1. As the figure indicates, there is flexibility in choosing the filter and smoother used to calculate the distributions needed for EM. In this work, we choose two different combinations, a particle filter and particle smoother (Sec. III-B) and a UKF and Unscented Rauch-Tung-Striebel smoother (Sec. III-C). We begin with a brief review of EM.
Fig. 1:

Generic EM-based framework for simultaneous localization and parameter estimation
A. Expectation Maximization
Consider the problem of identifying an unknown parameter for the nonlinear state space model
| (10) |
where is the state, is the observation, and w and v are process and observation noise terms of appropriate dimension. Our goal is to find a Maximum Likelihood (ML) estimate of the parameter from the data , given by
| (11) |
This optimization can only be solved in closed form in certain simple cases as pθ(YN ) is typically intractable. EM approaches this problem by defining a hidden (or latent) variable and moving towards the maximum of l(θ) = pθ(YN) through iterative optimization of a function 𝒬 given by
| (12) |
where is current estimate of the parameter. The calculation of is called the Expectation (E)-step at the ith iteration. It has been shown that any choice of such that also increases the original likelihood [19]. Thus, the E-step is followed by a Maximization (M)-step to produce the next estimate,
| (13) |
Following [20], we decompose (12) as
| (14) |
where
| (15a) |
| (15b) |
| (15c) |
To determine the distributions needed in (15), we turn to filtering and smoothing algorithms.
B. SMC-EM
Under SMC-EM, the filtering and smoothing are done using a particle filter and a particle smoother. While any particle-based scheme could be used (see, e.g., [14] for an overview of particle methods), for simplicity here we use a basic Sequential Importance Resampling (SIR) filter and smoother. Under this choice, the functions in (15) are approximated as
| (16a) |
| (16b) |
| (16c) |
where N is the number of time steps, M is the number of sampled particles used for approximating the distributions, and are the sampled particles. The weights in (16) are given by first determining the importance weight
| (17) |
The smoothed weight is determined using a backward recursion,
| (18) |
Finally, the are given by
| (19) |
C. U-EM
U-EM approximates the posterior distribution for the state of a dynamic system with a Gaussian. It uses an unscented transform, propagating a set of deterministically selected sigma points through the model to calculate the posterior mean and covariance. The function becomes
| (20) |
where Q is the covariance of the process noise, R is the covariance of the observation noise, P is the covariance of the state, P0 and m0 are the initial estimate of the state covariance and mean state, P0|N, m0|N are the smoothed estimates of the state covariance and mean state at the initial time, and tr denotes the trace operation.
Through the UKF and URTSS, the approximated posterior densities needed for the EM algorithm are
| (21a) |
| (21b) |
Further details about the unscented approach and the U-EM scheme can be found in [8], [14]. Note that since the motion model demonstrated in (1) is linear, we could use the Kalman Filter (KF) for the propagation step in the motion model as well. To maintain generality, throughout this paper we use the UKF.
IV. Demonstration and Analysis
In order to demonstrate and compare performance of the EM-based algorithms, we turn to physical simulation where the ground truth is known. We note that the use of simulations to validate algorithms is well-established in the SPT community [21]. We simulated SPT datasets according to the optical parameters and other fixed constants shown in Table I. These values were chosen to represent common experimental settings found in many SPT experiments. When using U-EM, the tuning parameters for the UKF were set to (α, κ, β) = (1, 0, 2).
TABLE I:
Parameter settings
| Symbol | Parameter | Values |
|---|---|---|
| D | Diffusion coefficient | 0.01 μm2/s |
| A | Stiffness coefficient | 1.0s−1 |
| Δt | Image period | 100 ms |
| δt | Shutter period | 10 ms |
| P | Number of pixels per squared image | 25 |
| Δx | Length of unit pixel | 100 nm |
| Δy | Width of unit pixel | 100 nm |
| λ | Emission wavelength | 540 nm |
| NA | Numerical aperture | 1.2 |
| G | Peak intensity gain (signal) | 100 |
| u | velocity term | 0.01 μm |
| Nbgd | Background noise | 10 |
The observation model was chosen to simulate the pixel-dependent noise characteristics on the sCMOS chip of a Hamamatsu ORCA Flash 4.0 camera, following the approach described in [9]. Typical images of a fluorescent particle’s trajectory, a single frame of the image sequence, and the gain and variance of the sCMOS pixels are shown in Fig. 2.
Fig. 2:

(top) Typical (left) trajectory and (right) acquired image in the simulations. (bottom) Camera readout noise Nbgd = 10, G = 100.
A. Estimation with a fixed data length
In this case, we focus on a typical case with a fixed image length of N = 100. We simulate 100 sample paths and corresponding image sequences and analyze them using SMC-EM and U-EM. 10 EM iterations were run under each EM based method (see Fig. 3 showing the evolution of the parameter estimates as a function of the EM iteration).
Fig. 3:

Boxplot of estimates as a function of EM iteration for (top) U-EM and (bottom) SMC-EM100.
The overall mean and standard deviation of the estimated parameters are summarized in Table II. These results indicate that both U-EM and SMC-EM have good performance. As the number of particles used in SMC-EM grows, the RMSE, parameter estimation variance, and parameter estimation bias all decrease. However, this comes at a cost in computation time.
TABLE II:
Parameter estimation with 100 images
| Method | D (μm2/s) | A (s−1) |
|---|---|---|
| U-EM | 0.008514 ± 0.00072991 | 1.01 ± 0.28134 |
| SMC-EM50 | 0.0080737 ± 0.00096272 | 0.99224 ± 0.2702 |
| SMC-EM100 | 0.0086592 ± 0.00087748 | 1.0164 ± 0.28005 |
| SMC-EM500 | 0.0092505 ± 0.00091714 | 1.0466 ± 0.29636 |
Correspondingly, the overall mean and standard deviation of the estimated position are summarized in Table III where performance is determined using the Root Mean Square Error (RMSE) between the true particle position and the mean of the smoothed distribution p(xt | YN) across an entire trajectory. In the table, SMC-EMM denotes an SMC-EM scheme using M sampled particles. As expected, these results show that the performance of SMC-EM depends strongly on the number of particles used. All schemes, however, show very good performance with a resolution far below the diffraction limit. These same results are shown as boxplots in Fig. 4.
TABLE III:
Localization performance with 100 images
| Method | RMSEx (nm) | RMSEy (nm) |
|---|---|---|
| U-EM | 8.6558 ± 0.9979 | 8.9193 ± 1.1069 |
| SMC-EM50 | 13.6751 ± 2.0489 | 13.4994 ± 1.8195 |
| SMC-EM100 | 10.6292 ± 1.3278 | 10.7201 ± 1.3692 |
| SMC-EM500 | 7.5029 ± 0.7495 | 7.6169 ± 0.7535 |
Fig. 4:

Boxplot of RMSE by U-EM and SMC-EM of (left, blue) x and (right, red) y.
A typical example of a trajectory estimation result by SMC-EM and U-EM shown in Fig. 5. For space reasons, only results in x are shown; results in y are similar.
Fig. 5:

Typical trajectory estimation result.
B. Computational complexity
Generally, the basic time complexity of SMC-EM is compared to for U-EM, where E is the total number of EM iterations, N is the image length, and M is the number of particles. It is clear that U-EM has a significant computational advantage. This reduction in complexity comes, of course, at the cost of generality in the posterior distribution describing the position in the particle at each time point since the UKF-URTSS approximates this distribution as a Gaussian while the particle-based approaches can represent other distributions [14].
Of course, complexity is a coarse metric; the actual computation time is also important. As part of this work, we explored the bottlenecks in computation and found that the main limitation was the calculation of the double integrals in the observation model (4). Therefore, we replaced the direct execution by a table lookup approach which guarantees an error < 10−3 for computing λp,t. To improve the SMC performance, we took advantage of parallel processing in the Matlab environment. The calculations were carried out on a 2.3 GHz Intel Core i5 running Mac OS 10.14.4. Fig. 6 shows the corresponding improvement in the runtime of the two SPT methods. Note that it takes only two seconds for U-EM to complete an analysis run with 100 images.
Fig. 6:

Runtime of different EM-based methods on single dataset at Nbgd = 10, G = 100 with image length of 100.
C. Estimation as a function of data length
In the SPT setting, the number of image frames available depends on a variety of factors including the frame rate, the intensity of the excitation, and the type of fluorescent label used. Data sets can range from the 10’s to 1000’s of frames. In this work, we explored different data lengths, from N = 10 to 1000 on log spacing. For each N, 100 datasets were simulated with parameter settings as Table I.
The parameter estimation and localization performance by U-EM, SMC-EM100, and SMC-EM500 are shown in Fig. 7. In this work, 10 EM iterations are enough for estimation convergence. Due to space limitations, only the results of RMSEx are presented here; results of RMSEy are similar. As expected, as the number of images increases, the final estimates have lower variance and bias for the parameters and lower RMSE. For SMC-EM, more images mainly contributes to a reduced variance, while the larger number of sampled particles mainly contributes to a closer median estimate.
Fig. 7:

Boxplot of final estimation by (top row) U-EM, (middle row) SMC-EM100, and (bottom row) SMC-EM500.
V. Conclusions
In this paper we described the application of two EM-based methods, SMC-EM and U-EM, to SPT data analysis, focusing on Ornstein-Uhlenbeck motion and sCMOS cameras. Our results indicate that U-EM has a significant advantage over SMC-EM in terms of computation time but that, with increasing number of particles in the Monte Carlo methods, SMC-EM provides more accurate estimation. We also explored the impact of data length on estimation performance with results showing that increasing the amount of data reduces both bias and variance. For future work, we plan to extend the application to 3-D SPT scenarios.
Acknowledgement
This work was supported in part by the NIH through 1R01GM117039-01A1.
References
- [1].von Diezmann A, Shechtman Y, and Moerner WE, “Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking,” Chemical Reviews, vol. 117, no. 11, pp. 7244–7275, Jun. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Shechtman Y, Sahl SJ, Backer AS, and Moerner W, “Optimal point spread function design for 3d imaging,” Physical Review Letters, vol. 113, no. 13, p. 133902, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Ma Y, Wang X, Liu H, Wei L, and Xiao L, “Recent advances in optical microscopic methods for single-particle tracking in biological samples,” Analytical and Bioanalytical Chemistry, pp. 1–19, 2019. [DOI] [PubMed] [Google Scholar]
- [4].Michalet X, “Mean square displacement analysis of single-particle trajectories with localization error: Brownian motion in an isotropic medium,” Physical Review E, vol. 82, no. 4, p. 041914, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Michalet X and Berglund AJ, “Optimal diffusion coefficient estimation in single-particle tracking,” Physical Review E, vol. 85, no. 6, p. 061916, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Digiacomo L, D’autilia F, Durso W, Tentori PM, Caracciolo G, and Cardarelli F, “Dynamic fingerprinting of sub-cellular nanostructures by image mean square displacement analysis,” Scientific Reports, vol. 7, no. 1, pp. 1–10, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Ashley TT and Andersson SB, “Method for simultaneous localization and parameter estimation in particle tracking experiments,” Physical Review E, vol. 92, no. 5, p. 052707, 2015. [DOI] [PubMed] [Google Scholar]
- [8].Lin Y and Andersson SB, “Simultaneous localization and parameter estimation via sigma point bsed em,” Proc. of IEEE Conference on Decision and Control, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Huang F, Hartwich TM, Rivera-Molina FE, Lin Y, Duim WC, Long JJ, Uchil PD, Myers JR, Baird MA, Mothes W et al. , “Video-rate nanoscopy using scmos camera–specific single-molecule localization algorithms,” Nature Methods, vol. 10, no. 7, pp. 653–658, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Babcock HP, “Multiplane and spectrally-resolved single molecule localization microscopy with industrial grade cmos cameras,” Scientific Reports, vol. 8, no. 1, pp. 1–8, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Beausang JF, Zurla C, Finzi L, Sullivan L, and Nelson PC, “Elementary simulation of tethered brownian motion,” American Journal of Physics, vol. 75, no. 6, pp. 520–523, 2007. [Google Scholar]
- [12].Calderon CP, “Motion blur filtering: A statistical approach for extracting confinement forces and diffusivity from a single blurred trajectory,” Physical Review E, vol. 93, no. 5, p. 53303, 2016. [DOI] [PubMed] [Google Scholar]
- [13].Zhang B, Zerubia J, and Olivo-Marin J-C, “Gaussian approximations of fluorescence microscope point-spread function models,” Applied Optics, vol. 46, no. 10, pp. 1819–1829, 2007. [DOI] [PubMed] [Google Scholar]
- [14].Särkkä S, Bayesian filtering and smoothing. Cambridge Univ. Press, 2013. [Google Scholar]
- [15].Anscombe FJ, “The transformation of poisson, binomial and negative-binomial data,” Biometrika, vol. 35, pp. 246–254, 1948. [Google Scholar]
- [16].Freeman MF and Tukey JW, “Transformations related to the angular and the square root,” Ann. Math. Statist, vol. 21, no. 4, pp. 607–611, 1950. [Google Scholar]
- [17].Gnedenko BV, Theory of probability. Routledge, 2017. [Google Scholar]
- [18].“optimal inversion of the generalized anscombe transformation for poisson-gaussian noise.” [DOI] [PubMed] [Google Scholar]
- [19].Dempster AP, Laird NM, and Rubin DB, “Maximum likelihood from incomplete data via the EM algorithm,” Journal of the Royal Statistical Society. Series B, vol. 39, no. 1, pp. 1–38, 1977. [Google Scholar]
- [20].Schön T, Wills A, and Ninness B, “System identification of nonlinear state-space models,” Automatica, vol. 47, no. 1, pp. 39–49, 2011. [Google Scholar]
- [21].Sage D, Pham T-A, Babcock H, Lukes T, Pengo T, Chao J, Velmurugan R, Herbert A, Agrawal A, Colabrese S et al. , “Super-resolution fight club: assessment of 2d and 3d single-molecule localization microscopy software,” Nature Methods, vol. 16, no. 5, pp. 387–395, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
