IMAGE REGISTRATION ERROR ANALYSIS WITH APPLICATIONS IN SINGLE MOLECULE MICROSCOPY

Abstract

This paper is concerned with assessing localization errors emanating from the image registration of two monochromatic fluorescence microscopy images. Assuming an affine transform exists between images, registration in this setting typically involves using control points to solve a multivariate linear regression problem; however with measurement errors existing in both sets of variables the use of linear least squares is inappropriate. It is shown that image registration is an errors-in-variable problem and as such the correct method is to use generalized least squares. Traditionally this requires the measurement errors to be independent and identically distributed (iid); an assumption that is rarely satisfied in practical situations. An extension of the multivariate generalized least squares estimator that allows non-iid noise is applied. The distributional properties of the estimators are used to derive localization errors emanating from the image registration process in terms of photon counts and experimental parameters.

1. INTRODUCTION

In its most general form, image registration is the process of transforming different data sets into a single coordinate system. It is of use in areas as diverse as computer vision, target tracking, medical imaging and in fluorescence microscopy, the setting for this paper.

Recent advances in fluorescence microscopy have made it possible to detect single molecules in a cellular environment, e.g. [1]. A typical goal of a fluorescence microscopy experimental set-up is to observe the relative positions of a collection of two different protein molecules, for example to see whether they colocalize e.g. [2]. Two different fluorescents that emit at different wavelengths are used to tag the proteins, one for each type of molecule. The two types of molecule are imaged separately, but simultaneously using a pair of filters suitable for the fluorescents. A pair of monochromatic images are formed. It is often the case that the pair of images can not be considered to use the same coordinate system and the goal is to register the images so that the positions of the molecules can be viewed in unison, e.g. [3]. To do so we consider an image to be a subset of the space ℝ². Suppose we have two images, ℐ₁ and ℐ₂, say, and there is some mapping T : ℐ₁ → ℐ₂ between the pair of images; the type of mapping assumed here is an affine transformation.

An affine transformation between two vector spaces is a linear transformation followed by a translation. When both vector spaces are ℝ^d then T : ℝ^d → ℝ^d is affine if there exists a matrix parameter A ∈ ℝ^d×d (the linear transformation), and a vector parameter t ∈ ℝ^d (the translation) such that for x ∈ ℝ^d, T : x → Ax + t. In this situation we are concerned with the case d = 2.

In practice the affine transformation that exists between a pair of images is not known and needs to be estimated experimentally. To do this it is necessary to have points of reference (control points) visible in both images. In fluorescence microscopy this is achieved through fiduciary markers such as small but bright fluorescent beads [2]. These objects emit photons across the visible spectrum resulting in them being observed through both filters and appearing in both images.

Suppose K control points are located in ℐ₁ at true locations $\{x_k^{(1)},k=1,\dots ,K\}$, and in ℐ₂ at true locations $\{x_k^{(2)},k=1,\dots ,K\}$, such that $x_k^{(2)}=T(x_k^{(1)})={\mathit{\text{Ax}}}_k^{(1)}+t$. In reality the position of these beads cannot be known exactly and must instead be measured. Consequently the true coordinates are not directly observed, but instead we observe coordinates that have been perturbed by some random additive noise. That is to say we observe $\{y_k^{(1)},k=1,\dots ,K\}\text{ and }\{y_k^{(2)},k=1,\dots ,K\}\text{ where }{y}_k^{(j)}=x_k^{(j)}+{\epsilon }_k^{(j)},k=1,\dots ,K$, j = 1,2, where ${\epsilon }_k^{(j)}\in {ℝ}^2$ is a random vector known as the measurement error with zero mean and covariance ${\mathrm{\Sigma }}_k^{(j)}$. Through the work of Ober and Ram [4] [5], in practice it is possible to determine the error covariance matrices $\{{\mathrm{\Sigma }}_k^{(1)},k=1,\dots ,K\}\text{ and }\{{\mathrm{\Sigma }}_k^{(2)},k=1,\dots ,K\}$, with dependency on the microscopy set up and the intensity of the objects being imaged. It is assumed throughout this paper that the localization errors are normally distributed with mean zero [6] [7].

Given that we are able to construct estimators  and t̂ of the transformation parameters A and t, registration of the two images can commence. In the molecular fluorescence microscopy setting we observe a molecule in ℐ₁ at point $y_a^{(1)}$, where $y_a^{(1)}=x_a^{(1)}+{\epsilon }_a^{(1)}$ (again the covariance of ${\mathrm{\epsilon }}_a^{(1)}$ is estimable from [4]). $x_a^{(1)}$ is the true positional vector of the molecule. We estimate its new coordinate (the registered position) in ℐ₂ as $Ây_a^{(1)}+\hat{t}$. A key question is; what is the error associated with the registered position in ℐ₂? Or equivalently to determine the covariance of the image registration error (IRE)

$${\nu }_a\equiv x_a^{(2)}-Ây_a^{(1)}-\hat{t},$$

where $x_a^{(2)}={\mathit{\text{Ax}}}_a^{(1)}+t$.

In Section 2 we formulate the image registration problem and show that parameter estimation is consistent with a multivariate errors-in-variable framework. This is followed by a discussion on the image registration error. In Section 3 is a discussion on the distributional properties of the measurement errors. Traditional linear least squares methods are biased for errors-in-variable models so it is necessary to use the generalized least squares method to estimate the affine transform parameters. These estimators and their associated distributional properties are presented in Section 4 and are used to determine the image registration error in Section 5.

2. THE IMAGE REGISTRATION PROBLEM

We define the following ℝ^2×K matrices for j = 1, 2

$$\begin{array}{c}{X}^{(j)}\equiv [x_1^{(j)},\dots ,x_K^{(j)}]\hfill \\ Y^{(j)}\equiv [y_1^{(j)},\dots ,y_K^{(j)}]\hfill \\ E^{(j)}\equiv [{\epsilon }_1^{(j)},\dots ,{\epsilon }_K^{(j)}],\hfill \end{array}$$

and further define the ℝ^4×K matrices

$$\begin{array}{ccc}X=\left[\begin{array}{c}{X}^{(1)}\hfill \\ X^{(2)}\hfill \end{array}\right],\hfill & Y=\left[\begin{array}{c}{Y}^{(1)}\hfill \\ Y^{(2)}\hfill \end{array}\right],\hfill & E=\left[\begin{array}{c}{E}^{(1)}\hfill \\ E^{(2)}\hfill \end{array}\right].\hfill \end{array}$$

With this notation the system of equations can be conveniently represented as the single matrix equation

$$Y=\alpha {\mathbf{1}}_K^T+\mathrm{\Lambda }{X}^{(1)}+E,$$ (1)

where α = [0^T, t^T]^T, Λ = [I₂, A^T]^T and 1_K is a column vector of length K with every element taking the value 1. The columns of X⁽¹⁾ are known as the independent variables and the columns of X⁽²⁾ are the dependent variables. Linear regression problems where both the independent and dependent variables contain measurement errors are known as errors-in-variable (EIV) models. Let E_k denote the kth column of E, then E_k is 4-D multivariate normally distributed with mean zero and covariance

$${\mathrm{\Sigma }}_k=\left[\begin{array}{cc}\hfill {\mathrm{\Sigma }}_k^{(1)}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\mathrm{\Sigma }}_k^{(2)}\hfill \end{array}\right],$$

for which we write $E_k\stackrel{\mathrm{d}}{=}{N}_2(\mathbf{0},{\mathrm{\Sigma }}_k)$.

2.1. Image registration error

Suppose we image a molecule in ℐ₁ with true position $x_a^{(1)}\in {ℐ}_1$. The true position of the molecule in ℐ₂, $x_a^{(2)}\in {ℐ}_2$ will be related to $x_a^{(1)}$ by the same affine transform, namely $x_a^{(2)}=A{x}_a^{(1)}+t$. Let $y_a^{(1)}=x_a^{(1)}+{\epsilon }_a^{(1)}$ be the measured position of the molecule in ℐ₁, where ${\epsilon }_a^{(1)}\stackrel{\mathrm{d}}{=}{N}_2(\mathbf{0},{\mathrm{\Sigma }}_a^{(1)})$.

Definition 2.1. The image registration error (IRE) ν_a = [ν_a,1, ν_a,2]^T associated with registering the molecule in ℐ₂ is defined as

$${\nu }_a\equiv x_a^{(2)}-Ây_a^{(1)}-\hat{t},$$ (2)

where  and t̂ are estimators of the matrix A and vector t, respectively.

Let us define the difference between the true and estimated values of the transform parameters as ΔA ≡ Â − A and Δt ≡ t̂ − t. It can be shown that

$${\nu }_a=-A{\epsilon }_a^{(1)}-\mathrm{\Delta }t-\mathrm{\Delta }A{x}_a^{(1)}-\mathrm{\Delta }A{\epsilon }_a^{(1)},$$

and hence

$$E\{{\nu }_a\}=-E\{\mathrm{\Delta }t\}-E\{\mathrm{\Delta }A\}{x}_a^{(1)},$$

where E{·} is the expectation operator. Let cov{υ} denote the covariance matrix of a vector υ.

Lemma 2.2. If  and t̂ are unbiased estimators and ${\epsilon }_a^{(1)}$ is independent of both ΔA and Δt, then E{ν_a} = 0 and

$$\text{cov}\{{\nu }_a\}=A{\mathrm{\Sigma }}_a^{(1)}{A}^T+E\{(\mathrm{\Delta }t+\mathrm{\Delta }A{x}_a^{(1)}){(\mathrm{\Delta }t+\mathrm{\Delta }A{x}_a^{(1)})}^T\}+E\{\mathrm{\Delta }A{\epsilon }^{(1)}{\epsilon }^{(1)T}\mathrm{\Delta }{A}^T\}.$$ (3)

To derive an expression for the second order moments of the IRE it is necessary to understand the distributional properties of the estimators ΔA and Δt.

3. MEASUREMENT ERRORS

In [4] detailed analytical expressions for the localization accuracy of a light emitting point source in a microscopy imaging experiment are derived allowing reliable estimates of the covariance matrices for the measurement errors. The Cramér-Rao lower bound (CRLB) theorem states:

Theorem 3.1. The covariance matrix of any unbiased estimator θ̂ of an unknown vector parameter θ is bounded from below by the inverse of the Fisher information matrix I(θ), i.e. cov(θ̂) ≥ I⁻¹(θ).

For the purposes of this paper we assume that the CRLB is attained. We consider the most general and suitable case for application purposes, whereby the object being imaged emits photons as a Poisson process, and there is an additional background Poisson noise process and Gaussian read out noise. Typically the beads have large photon emission rates in comparison to the background and readout noise processes. In such circumstances the Fisher information matrix for the estimator of the location parameter θ = (u, υ) of an object in ℐ_j can be considered to be of the form I(θ) = NZ^(j) where N ∈ ℤ⁺ is the photon count associated with the imaged object and Z^(j) is a ℝ^2×2 symmetric positive definite (SPD) matrix of known form.

The photon count associated with the kth bead in ℐ_j is labeled $N_k^{(j)}$. The noise terms ${\epsilon }_k^{(j)},k=1,\dots ,K$, j = 1,2, have covariance ${\mathrm{\Sigma }}_k^{(j)}=(1/N_k^{(j)}){\tilde{\mathrm{\Sigma }}}_0^{(j)}\text{ where }{\tilde{\mathrm{\Sigma }}}_0^{(j)}=Z^{(j)-1}$ is a SPD matrix and universal for all imaged points in ℐ_j. We now make the assumption that there is a linear relation between the brightness of the bead in each image, i.e. a bead that is bright in ℐ₁ is also bright in ℐ₂. Mathematically we say $N_k^{(2)}={\mathit{\text{cN}}}_k^{(1)}$ for all k = 1, …, K, where c > 0 is a constant of proportionality, universal for all control points. We define ${\overline{N}}^{(j)}\equiv (1/K){\displaystyle {\sum }_{k=1}^K}{N}_k^{(j)},\text{ and }{\mathrm{\Sigma }}_0^{(j)}\equiv (1/{\overline{N}}^{(j)}){\tilde{\mathrm{\Sigma }}}_0^{(j)}$, for j = 1, 2, then N̄⁽²⁾ = cN̄⁽¹⁾ and we have the situation where the covariance of the measurement errors are scalar multiples of a common matrix, i.e.

$${\mathrm{\Sigma }}_k=\eta k\left[\begin{array}{cc}\hfill {\mathrm{\Sigma }}_0^{(1)}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\mathrm{\Sigma }}_0^{(2)}\hfill \end{array}\right]$$

where ${\eta }_k={\overline{N}}^{(1)}/N_k^{(1)}\text{ and }{\mathrm{\Sigma }}_0^{(1)}\text{ and }{\mathrm{\Sigma }}_0^{(2)}$ are known.

4. GENERALIZED LEAST SQUARES WITH COVARIANCE WEIGHTING

Given the columns of E are iid with common covariance Σ₀ say, EIV models of type (1) are traditionally solved by minimizing the normalized residual sum of squares

$$r={\displaystyle \sum _{k=1}^K}{\upsilon }_k^T{\mathrm{\Phi }}^{-1}{\upsilon }_k$$ (4)

where ${\upsilon }_k=y_k^{(2)}-t-{\mathit{\text{Ay}}}_k^{(1)}$, Φ = UΣ₀U^T and U = [−A, I₂]. In the statistics literature, the values of A and t that minimize r are known as the generalized least squares (GLS) estimators [8]. In the engineering and numerical analysis literature total least squares (TLS) is the common approach where a different minimization problem is solved. Both estimators are identical to one another. Here we generalize (4) for the case where the columns of E no longer have common covariance.

Definition 4.1. The generalized least squares estimators  and t̂ are the values of A and t, respectively, that minimize the weighted residual sum of squares

$$r={\displaystyle \sum _{k=1}^K}{\upsilon }_k^T{\mathrm{\Phi }}_k^{-1}{\upsilon }_k,$$ (5)

where Φ_k = UΣ_kU^T.

Consider the multivariate EIV model (1), where the covariance of E_k is given by the SPD matrix Σ_k = η_kΣ₀ where $(1/K){\displaystyle {\sum }_{k=1}^K}{\eta }_k^{-1}=1$. For the estimators that minimize (5) and their asymptotic statistical properties we state the following two theorems. They are given full treatment in [9].

Theorem 4.2. Define the matrix $S\equiv (1/K){\displaystyle {\sum }_{k=1}^K}{\eta }_k^{-1}(y_k-\overline{y}){(y_k-\overline{y})}^T\mathit{\text{ where }}\overline{y}\equiv (1/K){\displaystyle {\sum }_{k=1}^K}{\eta }_k^{-1}{y}_k$. By considering V = GDG^T, the eigenvalue decomposition of matrix $V\equiv S{\mathrm{\Sigma }}_0^{-1}$, with D = diag{d₁, d₂, d₃, d₄} where d₁ ≥ … ≥ d₄ and GG^T = G^TG = I₄, then by making the partition

$$G=\left[\begin{array}{cc}{G}_{11}\hfill & G_{12}\hfill \\ G_{21}\hfill & G_{11}\hfill \end{array}\right],$$

the GLS estimator of A and t are given as

$$Â=G_{21}{G}_{11}^{-1},\hat{t}={\displaystyle \sum _{k=1}^K}{\eta }_k^{-1}{\mathit{\text{Uy}}}_k.$$

Theorem 4.3. Define ℝ² vector $\overline{x}\equiv (1/K){\displaystyle {\sum }_{k=1}^K}{\eta }_k^{-1}{x}_k^{(1)}$, ℝ^2×2 matrix $\mathrm{\Xi }\equiv (1/K){\displaystyle {\sum }_{k=1}^K}{\eta }_k^{-1}{x}_k^{(1)}{x}_k^{(1)T}$, matrix Ψ = Ξ − x̄x̄^T and matrix $\mathrm{\Theta }={\mathrm{\Psi }}^{-1}{({\mathrm{\Lambda }}^T{\mathrm{\Sigma }}_0^{-1}\mathrm{\Lambda })}^{-1}{\mathrm{\Psi }}^{-1}+{\mathrm{\Psi }}^{-1}$, then we have the following identities

$$\text{cov}\{\mathrm{\Delta }{t}_m,\mathrm{\Delta }{t}_n\}=K^{-1}(1+\overline{x}\mathrm{\Theta }{\overline{x}}^T){\mathrm{\Phi }}_{\mathit{\text{mn}}}$$ (6)

$$\text{cov}\{\mathrm{\Delta }{t}_l,\mathrm{\Delta }{a}_{\mathit{\text{mn}}}\}=K^{-1}{[\mathrm{\Theta }\overline{x}]}_n{\mathrm{\Phi }}_{\mathit{\text{ml}}}$$ (7)

$$\text{cov}\{\mathrm{\Delta }{a}_{\mathit{\text{mn}}},\mathrm{\Delta }{a}_{m\prime n\prime }\}=K^{-1}{\mathrm{\Theta }}_{\mathit{\text{mm}}\prime }{\mathrm{\Phi }}_{\mathit{\text{nn}}\prime }.$$ (8)

5. IMAGE REGISTRATION ERROR ANALYSIS

We now consider the asymptotic form of the IRE covariance matrix. Consider a molecule in ℐ₁ at point $y_a^{(1)}$, where $y_a^{(1)}=x_a^{(1)}+{\epsilon }_a^{(1)}$ with the measurement error having covariance Σ_a estimable from [4], and $x_a^{(1)}$ being the true positional vector. We estimate its new coordinate (the registered position) in ℐ₂ as $Ây_a^{(1)}+\hat{t}$. The IRE is defined in (2) and its covariance matrix given in (3). The individual elements of cov{ν_a} can now be calculated using (6), (7) and (8).

5.1. Approximating the image registration error

Let us consider some simplifying assumptions. It is common that the control points (beads) are randomly scattered throughout the image and as such we will model their location as a 2-D Gaussian distribution.

Lemma 5.1. Let control point positions $\{x_k^{(1)},k=1,\dots ,K\}$ be K realizations of a random variable 𝒳 = [𝒳₁, 𝒳₂]^T ∈ ℝ² where

$$𝒳\stackrel{\mathrm{d}}{=}{N}_2(\mu ,\mathrm{\Pi }),$$

then asymptotically x̄ = μ and Ψ = Π.

5.2. Assumptions

1. Consider the positions $\{x_k^{(1)},k=1,\dots ,K\}$ to be K realizations of a random variable 𝒳 = [𝒳₁, 𝒳₂]^T ∈ ℝ² where

   $$𝒳\stackrel{\mathrm{d}}{=}{N}_2(\mathbf{0},{\kappa }^2{I}_2).$$

   From Lemma 5.1 we have x̄ = 0 and Ψ = κ²I₂.

2. The affine transformation parameter A represents a rotation R combined with a magnification M. Consequently A would be of the form MR where RR^T = R^TR = I₂ and M = mI₂.

3. We model the measurement errors as $E_k\stackrel{\mathrm{d}}{=}{N}_4(\mathbf{0},{\mathrm{\Sigma }}_k)$ where

   $${\mathrm{\Sigma }}_k=\left(\begin{array}{cc}\hfill {\sigma }_{1,k}^2{I}_2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\sigma }_{2,k}^2{I}_2\hfill \end{array}\right)=\frac{{\overline{N}}^{(1)}}{N_k^{(1)}}\left(\begin{array}{cc}\hfill {\sigma }_1^2{I}_2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\sigma }_2^2{I}_2\hfill \end{array}\right)$$

   with the normalized variance terms ${\sigma }_j^{(2)}$ inversely proportional to the mean photon count, ${\sigma }_j^2={\zeta }^{(j)}/{\overline{N}}^{(j)}$. The terms ζ^(j) are known functions of the experimental parameters, including the Airy profile and photon wavelength [4].

Assumptions 1 – 3 give the identities x = 0, $\mathrm{\Phi }=(m^2{\sigma }_1^2+{\sigma }_2^2)I_2\text{ and }\mathrm{\Theta }={\kappa }^{-4}({\sigma }_1^2+m^2{\sigma }_2^2)I_2+{\kappa }^{-2}{I}_2$. We immediately conclude cov{Δt_l, Δa_ij} = 0 for l, i, j. Further to this $\text{cov}\{\mathrm{\Delta }{t}_i,\mathrm{\Delta }{t}_j\}=K^{-1}(m^2{\sigma }_1^2+{\sigma }_2^2){\delta }_{\mathit{\text{ij}}}$, where δ_ij is the Kronecker delta. cov{Δa_ij, Δa_i′j′} = 0 if i ≠ i′ and/or j ≠ j′. If i = i′ and j = j′, then given ${\kappa }^2\gg ({\sigma }_1^2+m^2{\sigma }_2^2)$ we can treat cov{Δa_ij, Δa_i′j′} ≈ 0. This is equivalent to stating that the spread of the control points is much greater than the individual measurement errors, a valid assumption in a microscopy setting.

Theorem 5.2. The covariance terms of the IRE can be approximated as

$$\text{cov}\{v_{a,i},v_{a,j}\}\approx {[A{\mathrm{\Sigma }}_a{A}^T]}_{\mathit{\text{ij}}}+K^{-1}({\sigma }_1^2{m}^2{\delta }_{\mathit{\text{ij}}}+{\sigma }_2^2).$$

If the covariance Σ_a is itself representable as ${\sigma }_a^2{I}_2$ then in terms of photon count the covariance terms become

$$\text{cov}\{v_{a,i},v_{a,j}\}\approx m^2{\sigma }_a^2{\delta }_{\mathit{\text{ij}}}+K^{-1}\left(m^2\frac{{\zeta }^{(1)}}{{\overline{N}}^{(1)}}{\delta }_{\mathit{\text{ij}}}+\frac{{\zeta }^{(2)}}{{\overline{N}}^{(2)}}\right).$$

6. CONCLUSION

When the control points have measurement error covariances that are known and some scalar multiple of a common SPD matrix, then the second order moments of the IRE have been derived. For microscopy applications we have shown there exists inverse dependencies on the mean photon counts of the control points, however there remains a theoretical lower bound to the variance or the IRE in either dimension of $m^2{\sigma }_a^2$. Examples and simulations are found in [9].