Troika of single particle tracking programing: SNR enhancement, particle identification, and mapping

Abstract

single particle tracking (SPT) techniques provide a microscopic approach to probe in vivo and in vitro structure and reactions. Automatic analysis of SPT data with high efficiency and accuracy spurs the development of SPT algorithms. In this perspective, we review a range of available techniques used in SPT analysis programs. In addition, we present an example SPT program step-by-step to provide a guide so that researchers can use, modify, and/or write a SPT program for their own purposes.

Introduction

Since single particle techniques were first applied decades ago,^1–10 scientists not only have cared about the ensemble average, but also have explored the dynamics of individual particles. Single particle tracking (SPT) in particular offers the potential to understand transport with unprecedented detail. For instance, ensemble chromatography has been explored at the single particle level,^11–13 distinct mechanisms of intracellular single protein and DNA transport have been identified,^14–17 diffusive displacements at the single particle level have permitted the study of fundamental properties such as non-Gaussian distributions,^18,19 and microrheology has combined single particle and macro-rheology techniques to characterize environmental factors in biological systems.^20–23 Thus, for a broad range of applications, SPT can provide transport dynamics free from ensemble averaging. However, the increased spatial and temporal resolutions intrinsic to SPT require the processing of large amounts of data that are often unique for each analyte. Therefore, customized and automated SPT programs that are efficient in terms of both accuracy and processing time are not only useful and elegant, but necessary.

Many commercial and free programs are available to analyse SPT data.^24–39 Existing programs are often designed and optimized for a specific condition. For instance, some programs focus on data with low signal-to-noise ratio (SNR),^27,33,40,41 some are designed for 3D particle tracking,^{27,34–37,42} and some are good for data with high particle density.^26,27 It is very convenient to use an established program if the analysis data format and the experimental system are similar, but this is unfortunately not often practical. Also, to customize an established program for an alternate purpose offers its own challenges. These realities make it safer or even easier in many cases to write your own program instead of using existing programs.

Finally, a customized program is needed if cutting-edge methods are applied in SPT. For example, several published works use a phase mask to change the depth dependence of the point spread function (PSF),^43–46 for example the double helix PSF.^43–45 If any custom PSFs are applied in SPT to improve axial resolution, a customized algorithm is required to identify recorded particles.^47,48 Also, long time and large spatially-resolved tracking can be achieved using four tetrahedrally located detectors to track particles.⁴⁹ In this case, a SPT program needs to consider the active movement of the focal volume. Under these and many other circumstances, researchers can apply some logic and ideas from established tracking programs but are more likely to require a customized option.

Therefore, we introduce the general logic and available methods of SPT programs by presenting a sample program and reviewing other SPT techniques that are used to study a wide variety of systems. We separate a SPT program into three important modules: SNR enhancement, particle identification, and particle mapping. The necessity, difficulty, and solutions for each module will be discussed in detail. To increase the efficiency and accuracy of identifying real particles, we increase the SNR of each frame first, which also increases the ability to analyse data taken at a wide range of experimental conditions. In addition, combined multiple criteria, which consider the local background, intensity, and shape of the PSF of the particle, maximize the accuracy of particle identification. Finally, we map particle trajectories by both applying the nearest neighbour algorithm to approach the global optimum (shortest total mapping distance) and assigning local crowded groups of particles by considering their total mapping distance to reach the local global optimal. The trajectories generated by our program can be directly used to calculate the diffusion coefficient using a maximum likelihood estimator (MLE). Our program applies and extends one or more general methods in each step in SPT analysis; and the review in each section provides an account for other available methods and more complicated analyses. By comparing the rate of incorrect mapping and calculated diffusion coefficient using simulated data under different scenarios, we concluded that these steps are valuable to be applied together in a SPT program. To aid with this goal, sample code and executables for each module are provided. The source code of the program written in MATLAB R2012a (with a user-friendly graphical interface as an option) is available at http://www.lrg.rice.edu/Content.aspx?id=96

Methods

Instrument and software

As needed and described below, both experimental and simulated data were employed for our algorithm development. Experimental data was collected on a home-built total internal reflectance fluorescence (TIRF) wide-field microscope. Samples were excited by a 532 nm solid state laser (Coherent, Compass 315M-100SL). The laser beam was passed through an acousto-optic modulator (IntraAction, 402AF1) controlled by frequency generators (Fluke, 271-U 115V) in a master-slave setup to synchronize the excitation at the same frequency as the collection rate.⁵⁰ The beam was expanded prior to focusing at the edge of a 1.45 NA, 100×, oil-immersion objective (Carl-Zeiss, alpha Plan-Fluar) for through-the-objective TIRF microscopy. At ~2 mW/cm², the TIRF excitation was attenuated to 1/e of its original power at a depth of ~85 nm. Emission was collected via epi-fluorescence and was separated from excitation with a dichroic mirror (Chroma, z532/633rpc) and notch filter (Kaiser, HNPF-532.0-1.0) and further filtered with a bandpass filter (Chroma, ET585/65m). The signal was detected on an electron-multiplied charge coupled device (EMCCD) (Andor, iXon 897) at an integration time of 10 ms and collection frequency of about 24 s⁻¹. The EMCCD has 512 × 512 pixels with pixel size corresponding to 64 nm in the acquired image. The full width at half maximum (FWHM) of a PSF was about 4–5 pixels. In each frame, usually less than 100 particles were recorded. Diffusion of particles was slowed down to less than 20 pixels between frames by controlling the viscosity of the solution (corresponding to: diffusion coefficient < 4 × 10⁷ nm²/s) to avoid cross talk among particles. The experimental data collected by the EMCCD was converted to a MATLAB compatible format and analysed by our program. Our SPT program is written using MATLAB R2012a.

Simulated data was generated by Erik Meijering for the Particle Tracking Challenge Contest Workshop in 2012.^24,51 We used simulated 2D Brownian diffusion data of vesicles (spherical shape). The simulated data had SNR of 7 and adjustable average numbers of particles of 100, 500 or 1000 in an image of 512 × 512 pixels.

All the calculations and tests were run on a personal computer (Lenovo Thinkpad T420) with an Intel i5, 2.5-GHz processor.

Sample preparation

All solutions were prepared with HyPure Molecular Biology grade water (Thermo Scientific). No. 1 glass coverslips (VWR, 22 × 22 mm) were cleaned in a TL1 solution (4% (v/v) H₂O₂ (Fisher Scientific) and 13% (v/v) NH₄OH (EMD Chemicals)) at 80 °C for 1.5 min and then plasma cleaned in O₂ for 2 min (Harrick Plasma, PDC-32G). Orange fluorescent FluoSpheres® beads (100 nm, max abs/em: 540/560 nm, Molecular Probes) were diluted to 1:1000 in solutions of 20% (w/w) methyloxypolyethylene glycol 5000 propionic acid N-succinimidyl ester (Fluka Analytical). Solutions were pipetted into the well of a custom silicon template (Grace Bio-Labs, 43018M) placed on a clean coverslip.

The DNA binding experiment was carried out with the binding of a dye labelled probe DNA (pDNA) to immobilized target DNA (tDNA) that has complementary sequence with the pDNA. The DNAs are from Eurofins MWG Operon. The tDNA is: biotin-5’-TTTCTTATGACAGAATGATTCAACTAACATACTTGA-CCACTCCCCACCCAC-AGCTTATGACAGAATGAT TCAACTAACATACTTTT. The pDNA is: Alexa532-5’-GTGGGTGGGGAGTGG. The substrate was co-grafted with an aqueous mixture of 25 % (m/m) methoxy-terminated N-succinimidyl polyethylene glycol 5000 (PEG-NS-5000, Sigma-Aldrich), 0.25 % (m/m) 5 kDa biotin-terminated PEG-NS (NOF Corporation, Japan), and 0.8 % (m/m) NaHCO₃ (Sigma-Aldrich) overnight then washed with water. A silicon flow cell (Grace Bio-labs) was attached to the biotinylated glass. The glass was incubated with a mixture of 0.25 mg/mL streptavidin (Invitrogen), 25 mM HEPES (pH ~ 7), and 40 mM NaCl for 2 min, and flushed with buffer solution (25 mM HEPES, 40 mM NaCl) three times. The biotin-tDNA (0.1 nM) in buffer solution was injected into the flow cell, incubated for 1 min, and was flushed with buffer solution three times again. Then the pDNA (1 nM) in buffer solution was injected into the flow cell for the fluorescent imaging of the binding of pDNA to the tDNA.

Definitions

We will define our terminology used in later discussion for convenience. The raw data is a sequence of 2D images recorded at an identical time lag. We call the sequence of 2D images a movie, and each image is referred to as a frame. The time lag between two consecutive frames is called the sampling time (dt). Later when we discuss mapping the same particles in consecutive frames, we will refer to the frame at earlier time as frame 1, and the latter one as frame 2. Each frame is a matrix of pixels. A pixel is the smallest information unit in our data, and records the signal intensity. Based on our algorithm, a group of pixels may be identified as a particle. A fitting region is a square region smaller than the total image that contains the particle’s PSF. The program isolates this region and applies the PSF position fitting algorithm to calculate the position of the particle to a subpixel level. We combine several criteria to identify the real particle; before all the criteria are finished, we will call regions selected to be considered possible particles, particle candidates. If an identified particle is a real particle, we will call it true particle, otherwise it is false particle. However, if a particle is true or false is mainly determined by the situation. For example, simulated data provide the only completely reliable means of testing for absolute accuracy in particle identification. When analysing real experimental data, visual inspection as a function of analyte concentration and experimental conditions is necessary. In this paper, the SNR of a particle is defined by equation (1)⁵²:

$$SNR=\frac{I_p-I_b}{\sigma }$$ (1)

Where I_p is the peak intensity, I_b is the average background value, and σ is the standard deviation of noise.

General steps

Our particle tracking program is composed of three separate steps (1, 2, 3) and one overall program (4); these steps can be used independently:

1. Increase SNR for each frame.

2. Identify particles in each frame, and calculate the particles’ positions, Gaussian radii, and intensities.

3. Map the same particles in consecutive frames.

4. Load the raw data, call steps 1–3 one by one, and use the calculated parameters in step 2 and the maps in step 3 to generate trajectories. Pass the generated trajectories to algorithms designed to calculate the diffusion coefficients.

Experimental Details

1. Increase SNR

Increasing the SNR of each frame is necessary if the SNR of the raw data is low, but also helps in general to improve the accuracy in particle identification. Many techniques have been presented to improve the SNR in single particle applications.^{27,33,40,41,53–55} Among them, Bayesian blinking and bleaching⁵³ and super-resolution optical fluctuation imaging⁵⁴ are two techniques of combining the information of multiple frames to increase the overall image resolution. However, they are not applicable in SPT due to the diffusion properties of particles. Applying an intensity threshold to the frame is a classic method to rule out background and noise and enhance PSFs,⁵⁵ but usually the intensity of particles and noise is not well separated, so finding a global threshold is impossible in case of a complicated background. Wavelet denoising is another approach to increase SNR of single molecule spectroscopy and particle imaging applications.^56,57 Olivo-Marin has developed an algorithm that uses the à trous wavelet transform to detect isolated bright spots.^27,41 The effect of this wavelet method is shown in Figure 1. Other methods include Laplacian-Gaussian masks,⁴⁰ which are good at edge detection and have been shown to be useful in intra-cellular tracking even when the images are extreme noisy.³³

Fig. 1.

Image of Rhodamine-labeled endosomes in a baby hamster kidney cell before (a) and after (b) applying spot detection using à trous wavelet transform to increase the SNR.⁴¹ Reprinted from Ref⁴¹. Copyright 2002 with permission from Elsevier.

In our sample program, the goal of increasing the SNR in each frame is to improve the efficiency and accuracy in the second step, particle identification. Particles in one frame can be classified into three general categories based on their SNR and their shortest distance from other particles: 1) locally isolated particles with high SNR (Figure 2a); 2) particles with high SNR but close to another particle (Figure 2b); 3) particles with low SNR (Figure 2c). Another case, which describes particles that are close together and with low SNR, is rare for images with lower or medium particle density and thus will not be discussed. Sergé’s exhaustive particle detection method²⁶ and Jaqaman’s time average methods²⁷ are designed to deal with this case in analysing images with high particle density. Figure 2 shows experimental examples of each category from the same frame obtained from measuring fluorescent polystyrene beads diffusing in water on plasma cleaned glass. Because the microscope is set to TIRF mode, even though these beads are identical, their positions within the exponentially decaying evanescent field vary, resulting in brightness variation, which in turn yields a different SNR for each particle in a frame.

Fig. 2.

Three categories of particles from the same frame. (a) Isolated particle with high SNR (142); (b) two particles close to each other (SNR ~ 50); (c) isolated particles with low SNR (5.6).

To correctly identify all the true particles, especially particles in category 3, we need to increase the SNR of each frame. Simply setting a smaller threshold does not work even in this simple case because it would cause noise or the edges of bright particles to be identified as false particles. Therefore, we apply an algorithm that can both smooth the noise and improve the PSF of dim particles. This method can increase the SNR of dim particles (e.g. Figure 2c) without significantly affecting the ability to distinguish particles that are close together (e.g. Figure 2b).

We increase the SNR of each frame by employing a local average method (Figure 3).^26,28,32,58 To achieve the minimal influence in distinguishing particles in category 2, the local average is confined to each pixel’s 8 nearest neighbours. An efficient way to calculate the local average for each pixel is to conduct a convolution in the Fourier domain. The raw data (Figure 3a) is convoluted with a 3 × 3 matrix of ones. In this process, the raw data is first transformed to the Fourier domain by the 2D Fourier transform function in MATLAB. Then, a 3 × 3 matrix of ones is expanded to the same size as the frame: the centre is a 3 × 3 matrix of ones, and other elements are zeros. After that, this additional expanded matrix is transformed to Fourier domain. The corresponding elements of the two Fourier domain matrices are multiplied to generate the resulting image in Fourier domain, and the inverse Fourier transform produces the image with increased SNR as indicated in Figure 3b.

Fig. 3.

Image before and after SNR enhancement. (a) The raw data of two dim particles (Figure 2c.), convolution with a 3 × 3 matrix of ones to generate figure b; (b) particles with higher SNR and better profile. The SNR increases from 5.6 to 13.1. The scale bar and colour map for the two figures are the same. The irregular shape is attributable to motion blur.

Comparing the original image (Figure 3a) to the image after increasing the SNR (Figure 3b), it is clearly observed that random noise is smoothed, and dim particles are qualitatively better resolved. Quantitatively, the algorithm can increase the SNR of a particle by 2 to 3 times, with only negligible broadening of the FWHM, (less than 1 pixel, shown in Table 1), as long as the FWHM of the particle is larger than 2 pixels, which is usually true in SPT experiments. Using larger than a 3 × 3 matrix of ones for the convolution potentially influences the FWHM of PSFs, and can make particles in Figure 2b indistinguishable. The effects on the SNR and FWHM of this algorithm are listed in Table 1, along with a comparison to one alternate method, spot detection.⁴¹ Our simple method increases the SNR by more than 2 and is similar to spot detection in its effect on particle FWHM. Also compared in Table 1 is relative computational processing time. Although spot detection removes noise completely, which makes the final SNR infinity, the computational time cost of spot detection is more than ten times longer than our method. (Our comparison is using MATLAB. The spot detection plugin of Icy written in Java is much faster.⁵⁹) Therefore, although for short-time applications, spot detection is optimal, for long-time tracking analysis a method such as our proposed local averaging technique is preferred.

Table 1.

Comparison of the effect on the average SNR and FWHM of particles and time cost per frame between our SNR enhancement and spot detection. Simulated vesicle diffusion data generated by Erik Meijering has been used.⁵¹ Average SNR and FWHM are calculated using tens of particles in the same frame (512 × 512 pixels).

	Average SNR	Average FWHM (pixels)	Time cost (s/frame)
Original PSF	8.3	3.4	0
Spot detection41	Infinity	4.1	1.02
Our method	18.7	4.2	0.08

2. Particle identification

From the denoised images, single particle locations are identified from the background. Threshold and local maximum are the key words in particle identification. The peak of a particle’s PSF is usually a local maximum. The local maximum can be defined as a pixel with higher signal than its nearest neighbour pixels²⁶ or if there is no brighter pixel within a certain distance.³² However, the local maximum can also pick a false positive point due to random bright noise or aggregation of particles.³² A simple way to solve this problem is to use a threshold.^25,26,32,60 For images with good SNR, this is usually enough, but for an uneven background, or if the intensities of particles have a broad distribution, a simple threshold is not sufficient. A more sophisticated discrimination method is discussed in Refs^28,32 that combine zero (m₀) and second (m₂) intensity moments to define the true particles and screen the false positives. In such a method, each particle candidate in a frame has its average intensity (m₀) and width of its PSF (m₂) calculated and mapped to m₂–m₀ coordinates as illustrated in Figure 4. The mathematic definitions of m₀ and m₂ are:

$$m_0={\mathrm{\Sigma }}_{i^2+j^2\le w^2}I(x+i,y+j)$$ (2)

$$m_2=\frac{1}{m_0}{\mathrm{\Sigma }}_{i^2+j^2\le w^2}(i^2+j^2)\times I(x+i,y+j)$$ (3)

Where I means intensity at each pixel; the value of W is related to the width of the PSF; (x, y) are the centre of each particle usually approximated by the local maximum; i, j are integers that indicate the pixels near the centre (x, y). As illustrated in Figure 4, real particles fall into a cluster (circled by dashed line) on the m₂-m₀ plot, and false particles such as noise or aggregated particles are outside the cluster. However, it is difficult to define the real particle cluster region. An optimized method has been suggested that uses a calibration sample to identify the real particle cluster.²⁸ This method is potentially useful in complicated systems like SPT in cells.

Fig. 4.

An example of mapping particle candidates to the (m₀, m₂) plane reproduced from Crocker’s work.²⁸ The condensed region in red dash-line circle represents the real particles. Reprinted from Ref²⁸. Copyright 1996 with permission from Elsevier.

In our sample program, we combined three criteria to identify all the real particles. First, most methods assume background and noise level are uniform in a frame,^25,26,60,61 but that is not always the case due to scattering or spatially heterogeneous samples. Instead, we provide the option to calculate localized background and noise level, as discussed below (2.1.). Next, in identifying particle candidates, it is common to consider the local maximum as a pixel with intensity larger than its 8 nearest neighbours.²⁶ Our method considers all the neighbours within a set distance to reduce the chance of identifying false particles, also discussed below (2.2.). And finally, in refining particle position, the centre of the PSF of each particle candidate will be calculated using a rapid but accurate algorithm, and the width of the PSF will be compared with Gaussian noise to further rule out the false particles (2.3.).

2.1. Generate localized threshold

For images having uneven background, as show in Figure 5a, it is necessary to calculate the threshold locally. As mentioned earlier, an uneven background may be caused by instrumental imperfections or sample heterogeneity. In Figure 5a, the background and average signal intensity on the lower right side of the image are larger than those on the left side. If a global threshold were applied to the entire image, true particles in the left side may be excluded. Therefore, we will calculate and apply a localized threshold to distinguish particles from background.

Fig. 5.

Determining the local background threshold using cumulative distributions of nearby pixel counts. (a) Image of dye-labelled probe DNA binding with unlabelled target DNA (static) on a plasma cleaned glass surface. (b) The cumulative distribution of pixel counts in a 50 × 50 pixel region (the black box in the image, ~100 × diffraction limit area of a single particle) is shown; the local background (bg) and standard deviation (sd) of the distribution is also calculated from the cumulative distribution. (c) The local background calculation is scanned over the whole image with a step interval of 25 pixels (~5 × diffraction limit) in both x and y directions. The background image with a bin size of 25 × 25 pixels is then generated with these local backgrounds. As illustrated by the boxes, the binned pixel red box is an average of the four 50 × 50 local background values indicated by the black box and the dotted boxes. This operation is applied to smooth the local background between adjacent binned pixels. The sd of the binned pixel is also calculated from the four local backgrounds around the pixel. (d) The threshold of the binned pixels is calculated to be bg + 3 × sd. The binned pixels are then un-binned back to the original resolution with a patch size of 25 × 25 pixels.

Our program calculates the local threshold for each 50 × 50 pixel region using a cumulative distribution. All pixels are considered; there is no separation of background from particles. The background can be considered as the average intensity of the selected region, but the average intensity can be highly biased by several bright particles. Alternatively, because the distribution of noise can be approximated as a Gaussian distribution,⁶² the background can also be considered as the Gaussian peak of the distribution of intensity. It is important to note that the use of a probability distribution (histogram) of intensity relies on a subjectively selected bin size – an improper bin size can mess up the distribution. A cumulative distribution (integrated probability distribution), instead, is an objective way to represent the distribution without introducing bin-size bias.⁶³ Figure 5b shows the cumulative distribution for the region in the black box in Figure 5a. The cumulative distribution is defined as equation (4):⁶³

$$P(I\prime >I)=\frac{1}{N}{\mathrm{\Sigma }}_{I\prime =I}^{I_{\mathit{\text{max}}}}n(I\prime )$$ (4)

Where N is the total number of pixels, and n(I’) is the number of pixels having intensity equal to I’. I is the intensity variable and I_max is the maximum intensity in the region. Ideally, we should fit the cumulative distribution to an error function to extract the background and the standard deviation of the noise, which is straightforward if we would like to calculate a global background for one entire image. Because we would like to calculate the background locally, the cumulative distribution/fitting process must be performed multiple times iteratively, resulting in more than 100 such calculations for each frame, requiring large computational times. Instead, we simply use the median of the cumulative distribution as local background (bg), and the intensity difference between 0.84 and 0.5 as the standard deviation (sd), (width between 0.84 and 0.5 of the cumulative distribution of a normal distribution equals the standard deviation of that normal distribution), as indicated in Figure 5b using dashed lines. The local background threshold is bg + 3 × sd, which excludes 99% of the noise.

We shift the box by 25 pixels horizontally or vertically each time so that each pixel is included in four regions, as illustrated in Figure 5c. In this way, each pixel will have a threshold that is averaged by its surrounding pixels. At the edges and corners, the 50 × 50 pixel region for calculating the local background and threshold is truncated, resulting in a minimum of a 25 × 25 pixels being used for those particular regions. In Figure 5c, the red 25 × 25 box is included in black box, dashed white box, dashed orange box, and dashed yellow box. We take the average of those four regions as the background and threshold for the red box region. Figure 5c shows the local background and Figure 5d shows the local background threshold. Clearly, brighter regions in Figure 5a have larger background and threshold in Figure 5c and 5d.

2.2. Identify particle candidates

A pixel is a local maximum if its intensity is larger than all the nearby pixels, as explained in Sergé’s method.²⁶ However, the definition of “nearby pixels” depends on the distance threshold. In our program, the local maximum is a pixel with an intensity larger than all the pixels within the distance threshold, and the intensities of all the pixels within the distance threshold should be larger than the local background threshold defined in 2.1. The selection of the distance threshold is related to the SNR and particles’ FWHM. Figure 6 explains the effect of using the appropriate distance threshold. The bright particle in Figure 6a has a FWHM ~ 5 pixels; if using a distance threshold equal to 1.5 pixels, which includes 8 nearby pixels, many false positive local maxima will be considered as particles, as shown in Figure 6b. As we increase the distance threshold to be 2 pixels, the number of false particles decreases significantly (Figure 6c). And when we set distance threshold to be 3 pixels, only the real particle is identified (Figure 6d). Setting the distance threshold as a tuneable parameter makes the local maximum algorithm an efficient method to identify particles in different experimental situations.

Fig. 6.

The effect of increasing distance threshold of local maximum on identification of true particles. (a) The area around the particles in the black box in Figure 5a. (b) Identified local maxima within distance threshold equal to 1.5 pixels (8 nearest neighbours, indicated in lower right corner). Each dot in the figure is a particle candidate. (c) Identified local maxima with distance threshold equal to 2 pixels (12 nearest neighbours). (d) Identified local maxima within distance threshold equal to 3 pixels (28 neighbours). The length scale for all four figures is the same as indicated in (a). The colour map (indicating intensity) for Figure (a) is 0 to 3000 counts and for Figure (b), (c), (d) is 0 to 500 counts.

2.3. Refine particle position

To achieve subpixel resolution of particle position, the centre of each PSF must be calculated. A Gaussian function fit to the PSF is the most widely used algorithm to refine particle position.^26,64–67 However, the fitting iteration and superfluous parameters make Gaussian fitting very inefficient, causing this step to become the computational bottleneck of a SPT program’s speed. In contrast, directly calculating the centroid of a PSF is very fast,^32,60 but its accuracy and reliability are unacceptable.^64,68 A couple of works have developed methods in between, which use a centroid to calculate the initial guess, and then apply another algorithm to update the position only once to reach a decent accuracy.^31,64 Radial symmetry is one of the best methods because its speed is 100 times faster than Gaussian fitting, and its accuracy is comparable to Gaussian fitting.⁶⁴

In our program, for each local maximum we found in 2.2, we pass a fitting region (the size of the fitting region depends on the PSF’s FWHM, but usually 2 – 4 times the FWHM is a good choice) with the maximum in the centre, to the radial symmetry function. The detail of the radial symmetry method is discussed in Parthasarathy’s work.⁶⁴ Basically, a PSF of a single particle is best represented by an Airy disk. Because of its radial symmetric property, the gradient of intensity at each position points to the centre of the PSF. Therefore, finding the centre of the PSF equals to finding the position where all the gradients at nearby positions point to. If we use a straight line to represent the gradient at each position, the centre of the PSF is where all the lines converge. However, because of the finite size of the pixel and the presence of noise, these lines cannot converge. But we can still locate the centre of the PSF by finding the position that has the smallest sum distance to all these lines.⁶⁴ This is the principle of the radial symmetry method. Using the calculated centre position of the particle, we will calculate the second momentum (m₂, using formula (3)) of the same fitting region. This m₂ will be compared with the m₂ of the same fitting region with Gaussian noise. Only if the m₂ of the particle is smaller than the 90 % of the m₂ of Gaussian noise, we will consider this particle as a real particle and record its positions.

3. Mapping particles

After particles are identified in each frame, their locations from subsequent frames must be linked together to form trajectories. Many algorithms have been proposed to determine the correspondence among particles in subsequent frames.^{27–29,31,32,35,39,58,60,69–72} These algorithms range from nearest-neighbour approaches,^28,31 and timely global methods,^39,58,69 to more complex methods that account for the challenges present in single molecule^29,60,71 and biological samples.^26,27,32,35

With samples that have low coverage, high SNR, and slow diffusion, a basic nearest neighbour approach is adequate to construct particle trajectories. A fundamental example has been presented in the highly-cited work of Crocker and Grier for tracking the diffusion of colloidal particles,²⁸ but is also commonly applied to tracking in biological environments.³¹ Nearest neighbour algorithms use distance as the only criterion to connect particle trajectories. Most of the time, when the number of particles varies from frame-to-frame, trajectories are either terminated or treated as new. The nearest neighbour method is simple and works well if the displacements are sufficiently smaller than the interparticle spacing with homogeneous diffusion (Figure 7a).^28,29

Fig. 7.

Comparison of nearest neighbour and dynamic search radius methods for forming trajectories. From Gao and Kilfoil,²⁹ example systems where (a) nearest neighbour is successful due to homogeneous diffusion and (b) fails due to heterogeneity and the dynamic radius method is applied. Green spheres are the particle positions in the first frame and the red circles are the particle positions in the next frame. The black circles represent the search radius for finding particles in subsequent frames. (c, d) Demonstration of the dynamic search radius method. (c) Blue circles represent particles assigned to a trajectory with the first pass of a small search radius and (d) increased search radius on second pass to assign trajectories with larger displacements. (e) Comparison of the probability distribution of average step size when using nearest neighbour (blue) and dynamic search radius (red) methods to track the simulated heterogeneous diffusion system represented in (b).²⁹ Reprinted and adapted with permission from Ref²⁹. Copyright 2009 Optical Society of America.

At the other end of the mapping spectrum are global methods that consider all possible connections for all particles throughout the data sequence using multiple-hypothesis tracking.⁶⁹ The number of non-conflicting trajectories is recursively maximized to find the optimal trajectories. A similar global method has been presented by Woell et al. that calculates a cost factor based on the displacement and time interval for all possible connections.⁵⁸ The trajectories are then selected based on the maximum likelihood. While these methods have very high accuracy, the computational cost can be too high to consider all possible connections, and hence is not efficient for tracking large numbers of particles across large numbers of frames. However, for more challenging trajectories containing photoblinking,^39,58 or merging and splitting behaviors,^27,39 global methods bear considerable advantages, and with optimized algorithms, the computational costs can be kept relatively low.^27,39,58

Overall, most methods take an approach between the simple, sample-limited nearest neighbour and the computationally-prohibitive, statistically-based global mapping. Several methods use a local search radius to find nearest neighbours and then implement further filtering and/or statistical cost factors to be able to analyse samples with high concentrations of particles, overlapping trajectories, diffusion heterogeneity, and particle disappearance that commonly occur in more complex samples.^{26,27,29,32,35,60,71} For example, Gao and Kilfoil describe a multipass tracking approach to form trajectories where heterogeneous diffusion in dense environments is present (Figure 7b).²⁹ A dynamic search radius is used where the local tracking method starts with a small search radius to form trajectories with slower diffusion (Figure 7c). Particles with identified connections at the smaller search radius are then removed from the analysis and a second pass with an increased search radius completes more trajectories (Figure 7d). This process is repeated so more and more trajectories are completed. Simulations showed that two diffusion populations could be resolved with the dynamic search radius as compared to only one population when a single search radius was used (Figure 7e). However, ignoring connected particles can generate incorrect mapping, which is why we consider all the particles in a local group together when there is conflict in mapping, as we describe in the next section.

Our sample mapping algorithm uses the local optimum to approach the global optimum using distance as the only criterion, as explained in Figure 8. Figure 8 shows the mapping method using two example consecutive frames. In Figure 8a, particles in two frames are overlapped to indicate their positions, and represent the different possible situations in particle mapping. For each particle in frame 1, our program searches for its neighbours in frame 2 within a searching distance that should be approximately three times of the average displacement between consecutive frames (Figure 8b). The searching distance can be estimated and tested by examining the mapping results between the first two frames using our program. For particle 1, the only neighbour is particle a, so particle 1 is connected to particle a as illustrated by a solid line in Figure 8c. For particle 2, two neighbours are found in frame 2, but particle b is its nearest neighbour, so particle 2 is connected to particle b with higher confidence, illustrated by a solid line, but also connected to particle c with lower confidence, illustrated by a dashed line in Figure 8c. If no neighbour is found, then no connection can be built. Particle 6 is an example. This means the particle may conduct photoblinking or motion/rotation out of focus, which will be addressed in section 4. In this way, all connections are built as depicted in Figure 8c.

Fig. 8.

Our sample mapping algorithm. The blue circles marked as 1–6 and red circles marked as a-e represent particles in frame 1 and frame 2, respectively. (a) Overlapping frame 1 with frame 2 to show particles’ relative positions. (b) Locating neighbours in frame 2 for each particle in frame 1 within searching distance. (c) Connections between particles in frame 1 and frame 2 based on Figure b. The nearest neighbour is connected by solid lines and the other neighbours are connected by dashed lines. Because both particle 2 and 3 claim particle b is their nearest neighbour, the global optimum of the isolated graph (2, 3, b, c) needs to be further inspected. (d) Searching for the global optimum of the isolated graph. The shortest total distance is the most likely global optimum of the isolated graph. The correct optimum connections should be 2-c, and 3-b, as shown in Figure d.

The connections in Figure 8c form several isolated graphs; and the mapping in each isolated graph should reach the global optimum – the shortest total distance. In case of the first (1 and a) and third (4, 5 and d, e) isolated graphs, the mapping (solid lines) is already the global optimum. However, for the second isolated graph (2, 3 and b, c), both particles 2 and 3 are connected to particle b, which conflicts to the criterion that each particle should only be connected to one particle in consecutive frames. In this case, the global optimum is searched by comparing all the mapping options, and the one with the shortest total distance gives the global optimum, as shown in Figure 8d.

4. Trajectory storage and analysis

Using the previously described parameters (centre, particle connections, etc.) in each frame and particle connections for consecutive frames, the trajectory of each particle is recorded in a 3 dimensional matrix. Its 3 dimensions represent time, parameters (including centre and width of particles), and particle number. After particle mapping between the consecutive frames, there may be terminated particles (particles not connected to any particle in the next frame) and new particles (particles not connected to any particle in the previous frame) in each frame. Terminated particles in frame t will be compared with new particles in frame t+2 and will be connected if their distance is within 1.4 times the searching distance (because average displacement increases as the square root of time lag) to include cases of short-time photobinking, moving out of focus, or simply moving too close to another particle.

From the obtained matrix of trajectories, different analysis methods can be applied. Commonly, the average diffusion coefficient (D) of the particles is calculated. A variety of methods for determining the diffusion coefficient in SPT have been presented.^73–78 Mean square displacement (MSD) analysis is the most commonly used and simplest method, where the diffusion coefficient is proportional to a linear fit of the MSD vs. time lag for systems exhibiting Brownian diffusion, according to Einstein’s equation, 〈r²〉 = 4Dt.^73–76 While MSD analysis is rather straightforward, problems arise when noise is present or trajectories are short due to photoblinking, photobleaching, and translation or rotation out of the collection plane. Therefore, the reliability and improvement of the MSD method have been further investigated to optimize analysis of experimental data.

In regards to optimally long trajectories in the presence and absence of noise, Michalet and Berglund have presented the optimized least-squares fit (OLSF)⁷⁷ and maximum likelihood estimator⁷⁸ methods, respectively. OLSF iteratively estimates the number of MSDs that should be used for the best fitting of the D, and MLE searches for the D that maximize the multivariable Gaussian function based on the trajectory.⁷⁹ Overall, both methods were shown to reach the Fisher information theoretical limit of relative standard deviation (RSD) for simulated trajectories.

Further challenges in MSD analysis arise when trajectories are short or there are lapses in the location of the particle, which occurs commonly in ‘real’ systems that include photoblinking, photobleaching, or translation/rotation out of the observation volume. To accurately and precisely calculate D from short trajectories containing intermittencies, Shuang, et al. developed two MLE-based methods that were optimized for trajectories exhibiting different noise levels (as quantified by x = σ²/(Ddt)).^80,81 The precision of the presented MLE methods were less sensitive to the photoblinking rate (defined as the relative number of frames with a lapse in location divided by the total number of frames) compared to the OLSF method (Figure 9) and better than Berglund’s MLE method whose likelihood function cannot be computed in the presence of lapses. Overall, the two methods, MLE(1) and MLE(2), were shown to precisely estimate D from short trajectories (<100 time steps) exhibiting intermittency.⁸⁰

Fig. 9.

Comparison of the precision between (a) OLSF and (b) MLE(2) in the presence of photoblinking. The colour represents the RSD of the calculated D. MLE(2) is much less influenced by photoblinking.⁸⁰ Reprinted with permission from Ref⁸⁰. Copyright 2013 American Chemical Society.

Performance

To test the performance of our entire SPT program and the importance of the individual steps, we used simulated data generated by Erik Meijering for the Particle Tracking Challenge Contest Workshop in 2012.^24,51 For each particle density, we used 10 simulated frames to calculate the average rate of incorrect mapping and the error bar (the standard deviation of the rate of incorrect mapping). The results are checked using the ground truth provided by the Particle Tracking Challenge Contest Workshop. As indicated in Figure 10a, as particle density increases, the rate of incorrect mapping increases (open circles, distance threshold = 2 pixel). We have also shown that if one omits individual steps addressed above, such as determining the local global optimum (open squares), or if one chooses an improper distance threshold (distance threshold = 1 pixel) to select particle candidates (open triangle), or both (cross marker), the result is an increase in incorrect particle mapping. Figure 10a also shows that an improper distance threshold has a major influence when particle density is low, and the local global optimum is more important when the particle density is high.

Fig. 10.

The performance of our entire example SPT program as particle density increases. (a) The rate of incorrect mapping as a function of particle density and influenced by optimized algorithms. The x-axis indicates particle density in an image, which is number of particles divided by total number of pixels (512 × 512). Open circles represent using all the techniques in our SPT program under optimized parameters; open squares indicate the performance without checking the local global optimum in the mapping step; open triangles show the effect of using improper distance threshold to search for particle candidates; the cross marker represents the performance combining the improper distance threshold and not using global optimum. The rate of incorrect mapping is defined as number of incorrect mapping divided by number of all the correct mapping. The colour of each point matches the box colour in figure b. The lines provide a visual guide. The SNR is 7. (b) Sample images at different particle density for a 100 × 100 pixel area. The red circles indicate calculated positions by our particle identification program under optimum conditions, corresponding to the open circles in a.

Image examples at three particle densities are included in Figure 10b, which also illustrate our program’s ability to identify most of the particles and to correctly calculate their positions for different particle densities. Under higher particle density, there is a larger chance for particles to have incorrect assignments, so neglecting the global optimum causes more incorrect mapping. And under low particle density, the algorithm is more likely to identify a fake particle in the particle-free space and cause incorrect mapping.

The influence of incorrect mapping on the diffusion coefficient is summarized in Table 2. Trajectories under the different conditions described in Figure 10a are used to calculate the average value of diffusion coefficients (D) and the standard deviation (σ), with arbitrary units. Each D and σ can be compared with the D and σ calculated from the ground truth trajectories. Clearly, the calculated D and σ are more accurate and precise when all steps are used than when omitting the global optimum or using improper distance threshold. The decrease of both accuracy and precision is positively related to the rate of incorrect mapping. Overall, these comparisons have shown the value of employing one or more optimization steps together in a SPT program.

Table 2.

Averaged diffusion coefficient (D) and the standard deviation (σ) for different particle density at different fitting conditions corresponding to Figure 10a. (NGO: no global optimum; IDT: improper distance threshold; NGO&IDT: no global optimum and improper distance threshold)

Particle	density	NGO	IDT	NGO& IDT	All steps	Truth
Low	D(a.u.)	1.91	2.11	2.05	1.92	1.90
	σ(a.u.)	0.74	1.01	0.92	0.77	0.74
Mid	D(a.u.)	2.12	2.13	2.17	2.02	1,96
	σ(a.u.)	0.90	0.89	0.96	0.74	0.67
High	D(a.u.)	2.40	2.33	2.45	2.25	2.00
	σ(a.u.)	1.21	1.11	1.34	1.02	0.72

Conclusions

Today’s advances in SPT are spurred by the strong demand for trustable transport data analysis. As SPT techniques become increasingly powerful and popular, reliable, approachable SPT programs become important and necessary. Without an automatic SPT program, it is difficult, if not impossible, to make any quantitative analysis on SPT data. An efficient and accurate SPT program provides fast feedback and reliable analysis of even complicated experiments in biological systems. Many of the techniques are developed from other fields such as image processing, astronomy, computer science, etc..

Due to the complexity and diversity in SPT experiments, in the authors’ opinion, it is not likely to have a general SPT program that works for all cases. Higher analysing speed and more reliability are still future directions in program development, and could be obtained by using a faster fitting algorithm,⁶⁴ or by using the GPU to speed up the calculation.⁸² Applying cutting-edge techniques in your own program also requires the basic knowledge about SPT fundamentals. We predict that with the fast development of personal computer and SPT algorithm development, synchronous data analysis with SPT measurement will probably become a standard technique in the near future, which will increase the overall productivity of SPT research.