A Purely Algebraic Justification of the Kabsch-Umeyama Algorithm

Abstract

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two matrices of the same order. Over past decades, the algorithm of choice for solving this problem has been the Kabsch-Umeyama algorithm, which is effectively no more than the computation of the singular value decomposition of a particular matrix. Its justification, as presented separately by Kabsch and Umeyama, is not totally algebraic since it is based on solving the minimization problem via Lagrange multipliers. In order to provide a more transparent alternative, it is the main purpose of this paper to present a purely algebraic justification of the algorithm through the exclusive use of simple concepts from linear algebra. For the sake of completeness, a proof is also included of the well known and widely used fact that the orientation-preserving rigid motion problem, i.e., the least-squares problem that calls for an orientation-preserving rigid motion that optimally aligns two corresponding sets of points in d−dimensional Euclidean space, reduces to the constrained orthogonal Procrustes problem.

1. Introduction

In the orthogonal Procrustes problem [1, 2], given real matrices P and Q of size d × n, the problem is that of finding a d × d orthogonal matrix U that minimizes ∥U Q − P∥_F, where ∥ · ∥_F denotes the Frobenius norm of a matrix. On the other hand, in the constrained orthogonal Procrustes problem [3–5], the same function is minimized but U is constrained to be a rotation matrix, i.e., an orthogonal matrix of determinant 1. By letting p_i, q_i, i = 1…, n, be the vectors in ${ℝ}^d$ that are the columns from left to right of P and Q, respectively, since clearly $\Vert UQ-P{\Vert }_F^2={\sum }_{i=1}^n{\Vert U{q}_i-p_i\Vert }^2$, where ∥ · ∥ denotes the d−dimensional Euclidean norm, then an alternative formulation of the two problems above is that of finding an orthogonal matrix U (of determinant 1 for the constrained problem) that minimizes ${\sum }_{i=1}^n{\Vert U{q}_i-p_i\Vert }^2$. We note that minimizing matrices do exist for the two problems as the function being minimized is continuous and both the set of orthogonal matrices and the set of rotation matrices are compact (in some topology). Finally, in the same vein, another problem of interest is the orientation-preserving rigid motion problem which is that of finding an orientation-preserving rigid motion ϕ of ${ℝ}^d$ that minimizes ${\sum }_{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2$. An affine linear function $\varphi ,\varphi :{ℝ}^d\to {ℝ}^d$, is a rigid motion of ${ℝ}^d$ if it is of the form $\varphi (q)=Uq+t\text{for}q\in {ℝ}^d$ , where U is a d × d orthogonal matrix, and t is a vector in ${ℝ}^d$. The rigid motion ϕ is orientation preserving if det(U) = 1, i.e., the determinant of U equals 1. With $\overline{p},\overline{q}$ denoting the centroids of {p_i}, {q_i}, respectively, as will be shown in Section 3 of this paper, this problem can be reduced to the constrained orthogonal Procrustes problem by translating {p_i}, {q_i} to become $\left\{p_i-\overline{p}\right\},\left\{q_i-\overline{q}\right\}$, respectively, so that the centroid of each set becomes $0\in {ℝ}^d$.

With P, Q, p_i, q_i, i = 1, …, n, as above, in this paper we focus our attention mostly on the constrained orthogonal Procrustes problem, and therefore wish to find a d × d rotation matrix U that minimizes ${\sum }_{i=1}^n{\Vert U{q}_i-p_i\Vert }^2$.

With this purpose in mind, we rewrite ${\sum }_{i=1}^n{\Vert U{q}_i-p_i\Vert }^2$ as follows, where given a square matrix R, tr(R) stands for the trace of R.

$$\begin{gathered} \sum _{i=1}^n{\Vert U{q}_i-p_i\Vert }^2 \\ =\sum _{i=1}^n{\left(U{q}_i-p_i\right)}^T\left(U{q}_i-p_i\right)=\text{tr}\left({(UQ-P)}^T(UQ-P)\right) \\ =\text{tr}\left(\left(Q^T{U}^T-P^T\right)(UQ-P)\right)=\text{tr}\left(Q^{T}Q+P^{T}P-Q^T{U}^{T}P-P^{T}UQ\right) \\ =\text{tr}\left(Q^{T}Q\right)+\text{tr}\left(P^{T}P\right)-2\text{tr}\left(P^{T}UQ\right). \end{gathered}$$

Since only the third term in the last line above depends on U, it suffices to find a d × d rotation matrix U that maximizes tr(P^T U Q). Since tr(P^T U Q) = tr(U QP^T) (note in general tr(AB) = tr(BA), A an n × d matrix, B a d × n matrix), denoting the d × d matrix QP^T by M, this problem is equivalent to finding a d × d rotation matrix U that maximizes tr(U M), and it is well known that one such U can be computed from the singular value decomposition of M [3–5]. This is done with the Kabsch-Umeyama algorithm [3–5] (see Algorithm Kabsch-Umeyama below, where diag{s₁, …, s_d} is the d × d diagonal matrix with numbers s₁, …, s_d as the elements of the diagonal, in that order running from the upper left to the lower right of the matrix). A singular value decomposition (SVD) [6] of M is a representation of the form M = V SW^T, where V and W are d × d orthogonal matrices and S is a d × d diagonal matrix with the singular values of M, which are nonnegative real numbers, appearing in the diagonal of S in descending order, from the upper left to the lower right of S. Finally, note that any matrix, not necessarily square, has a singular value decomposition, not necessarily unique [6].

Algorithm Kabsch-Umeyama has existed for several decades [3–5], however the known justifications of the algorithm [3–5] are not totally algebraic as they are based on exploiting the optimization technique of Lagrange multipliers. It is the main purpose of this paper to justify the algorithm in a purely algebraic manner through the exclusive use of simple concepts from linear algebra. This is done in Section 2 of the paper. Finally, we note that applications of the algorithm can be found, notably in the field of functional and shape data analysis [7, 8], where, in particular, the shapes of two curves are compared, in part by optimally rotating one curve to match the other.

2. Algebraic Justification of the Kabsch-Umeyama Algorithm

We justify Algorithm Kabsch-Umeyama using exclusively simple concepts from linear algebra, mostly in the proof of the following useful proposition. We note that most of the proof of the proposition is concerned with proving (3) of the proposition. Thus, it seems reasonable to say that any justification of the algorithm that requires the conclusion in (3) but lacks a proof for it, is not exactly complete. See page 47 of the otherwise excellent thesis in [9] for an example of this situation. See [10] for an outline of this dissertation.

Proposition 1:

If D = diag{σ₁, …, σ_d}, σ_j ≥ 0, j = 1, …, d, and W is a d × d orthogonal matrix, then

1. $\text{tr}(WD)\le {\sum }_{j=1}^d{\sigma }_j$.

2. If B is a d × d orthogonal matrix, S = B^T DB, then tr(W S) ≤ tr(S).

3. If det(W) = −1, σ_d ≤ σ_j, j = 1, …, d−1, then $\text{tr}(WD)\le {\sum }_{j=1}^{d-1}{\sigma }_j-{\sigma }_d$.

Proof:

Since W is orthogonal and if W_kj, k, j = 1, …, d, are the entries of W, then, in particular, W_jj ≤ 1, j = 1, …, d, so that $\text{tr}(WD)={\sum }_{j=1}^d{W}_{jj}{\sigma }_j\le {\sum }_{j=1}^d{\sigma }_j$ , and therefore statement (1) holds.

Accordingly, assuming B is a d × d orthogonal matrix, since BW B^T is also orthogonal, it follows from (1) that $\text{tr}(WS)=\text{tr}\left(W{B}^{T}DB\right)=\text{tr}\left(BW{B}^{T}D\right)\le {\sum }_{j=1}^d{\sigma }_j=\text{tr}(D)=\text{tr}(S)$ , and therefore (2) holds.

If det(W) = −1, we show next that a d × d orthogonal matrix B can be identified so that with $\overline{W}=B^{T}WB$ , then $\overline{W}=\left(\begin{array}{cc}{W}_0& O\\ O^T& -1\end{array}\right)$ , W₀ interpreted as the upper leftmost d − 1 × d − 1 entries of $\overline{W}$ and as a d − 1 × d − 1 matrix as well; O interpreted as a vertical column or vector of d − 1 zeroes.

With I as the d × d identity matrix, then det(W) = −1 implies det(W + I) = −det(W)det(W + I) = −det(W^T)det(W + I) = −det(I + W^T) = −det(I + W) which implies det(W + I) = 0 so that x ≠ 0 exists in ${ℝ}^d$ with Wx = −x. It also follows then that W^TWx = W^T (−x) which gives x = −W^Tx so that W^Tx = −x as well.

Letting b_d = x, vectors b₁, …, b_d−1 can be obtained so that b₁, …, b_d form a basis of ${ℝ}^d$, and by the Gram-Schmidt process starting with b_d, we may assume b₁, …, b_d form an orthonormal basis of ${ℝ}^d$ with Wb_d = W^Tb_d = −b_d. Letting B = (b₁, …, b_d), interpreted as a d × d matrix with columns b₁, …, b_d, in that order, it then follows that B is orthogonal, and with $\overline{W}=B^{T}WB$ and W₀, O as previously described, noting $B^{T}W{b}_d=B^T\left(-b_d\right)=\left(\begin{array}{c}O\\ -1\end{array}\right)$ and $b_d^{T}WB={\left(W^T{b}_d\right)}^{T}B={\left(-b_d\right)}^{T}B=\left(O^T-1\right)$, then $\overline{W}=\left(\begin{array}{cc}{W}_0& O\\ O^T& -1\end{array}\right)$. Note $\overline{W}$ is orthogonal and therefore so is the d − 1 × d − 1 matrix W₀.

Let S = B^T DB and write $S=\left(\begin{array}{ll}{S}_0\hfill & a\hfill \\ b^T\hfill & \gamma \hfill \end{array}\right)$, S₀ interpreted as the upper leftmost d − 1 × d − 1 entries of S and as a d − 1 × d − 1 matrix as well; a and b interpreted as vertical columns or vectors of d − 1 entries, and γ as a scalar. Note $\text{tr}(WD)=\text{tr}\left(B^{T}WDB\right)=\text{tr}\left(B^{T}WB{B}^{T}DB\right)=\text{tr}(\overline{W}S)$, so that $\overline{W}S=\left(\begin{array}{cc}{W}_0& O\\ O^T& -1\end{array}\right)\left(\begin{array}{cc}{S}_0& a\\ b^T& \gamma \end{array}\right)=\left(\begin{array}{cc}{W}_0{S}_0& W_{0}a\\ -b^T& -\gamma \end{array}\right)$ gives tr(W D) = tr(W₀S₀) − γ.

We show tr(W₀S₀) ≤ tr(S₀). For this purpose let $\widehat{W}=\left(\begin{array}{ll}{W}_0\hfill & O\hfill \\ O^T\hfill & 1\hfill \end{array}\right)$ W₀ and O as above. Since W₀ is orthogonal, then clearly $\widehat{W}$ is a d × d orthogonal matrix, and by (2), $\text{tr}(\widehat{W}S)\le \text{tr}(S)$ so that $\widehat{W}S=\left(\begin{array}{cc}{W}_0& O\\ O^T& 1\end{array}\right)\left(\begin{array}{cc}{S}_0& a\\ b^T& \gamma \end{array}\right)=\left(\begin{array}{cc}{W}_0{S}_0& W_{0}a\\ b^T& \gamma \end{array}\right)$ gives $\text{tr}\left(W_0{S}_0\right)+\gamma =\text{tr}(\widehat{W}S)\le \text{tr}(S)=\text{tr}\left(S_0\right)+\gamma$. Thus, tr(W₀S₀) ≤ tr(S₀).

Note tr(S₀) + γ = tr(S) = tr(D), and if B_kj, k, j = 1, …, d are the entries of B, then $\gamma ={\sum }_{k=1}^d{B}_{kd}^2{\sigma }_k$, a convex combination of the σ_k’s, so that γ ≥ σ_d. It then follows that $\text{tr}(WD)=\text{tr}\left(W_0{S}_0\right)-\gamma \le \text{tr}\left(S_0\right)-\gamma =\text{tr}(D)-\gamma -\gamma \le {\sum }_{j=1}^{d-1}{\sigma }_j-{\sigma }_d$, and therefore (3) holds. □

Finally, the following theorem, a consequence of Proposition 1, justifies the Kabsch-Umeyama algorithm.

Theorem 1:

Given a d × d matrix M, let V, S, W be d × d matrices such that the singular value decomposition of M gives M = V SW^T. If det(VW) > 0, then U = WV^T maximizes tr(UM) over all d × d rotation matrices U. Otherwise, if det(VW) < 0, with $\tilde{S}=\text{diag}\left\{s_1,\dots ,s_d\right\},s_1=\dots =s_{d-1}=1,s_d=-1$, then $U=W\tilde{S}{V}^T$ maximizes tr(UM) over all d × d rotation matrices U. □

Proof:

Let σ_j, j = 1, …, d, σ₁ ≥ σ₂ ≥ … ≥ σ_d ≥ 0, be the singular values of M, so that S = diag{σ₁, …, σ_d}.

Assume det(VW) > 0. If U is any rotation matrix, then U is orthogonal. From (1) of Proposition 1 since W^T UV is orthogonal, then $\text{tr}(UM)=\text{tr}\left(UVS{W}^T\right)=\text{tr}\left(W^{T}UVS\right)\le {\sum }_{j=1}^d{\sigma }_j$.

On the other hand, if U = WV^T, then U is clearly orthogonal, det(U) = 1, and $\text{tr}(UM)=\text{tr}\left(W{V}^{T}VS{W}^T\right)=\text{tr}\left(WS{W}^T\right)=\text{tr}(S)={\sum }_{j=1}^d{\sigma }_j$.

Thus, U = WV^T maximizes tr(UM) over all d × d rotation matrices U.

Finally, assume det(VW) < 0. If U is any rotation matrix, then U is orthogonal and det(U) = 1. From (3) of Proposition 1 since W^T UV is orthogonal and det(W^T UV) = −1, then $\text{tr}(UM)=\text{tr}\left(UVS{W}^T\right)=\text{tr}\left(W^{T}UVS\right)\le {\sum }_{j=1}^{d-1}{\sigma }_j-{\sigma }_d$.

On the other hand, if $U=W\tilde{S}{V}^T$, then U is clearly orthogonal, det(U) = 1, and $\text{tr}(UM)=\text{tr}\left(W\tilde{S}{V}^{T}VS{W}^T\right)=\text{tr}\left(W\tilde{S}S{W}^T\right)=\text{tr}(\tilde{S}S)={\sum }_{j=1}^{d-1}{\sigma }_j-{\sigma }_d$.

Thus, $U=W\tilde{S}{V}^T$ maximizes tr(UM) over all d × d rotation matrices U. □

3. Reduction of the Orientation-Preserving Rigid Motion Problem to the Constrained Orthogonal Procrustes Problem

Although not exactly related to the main goal of this paper, for the sake of completeness, we show the orientation-preserving rigid motion problem reduces to the constrained orthogonal Procrustes problem. For this purpose, let $\overline{q}$ and $\overline{p}$ denote the centroids of the sets ${\left\{q_i\right\}}_{i=1}^n$ and ${\left\{p_i\right\}}_{i=1}^n$ in ${ℝ}^d$, respectively:

$$\overline{q}=\frac{1}{n}\sum _{i=1}^n{q}_i \text{and} \overline{p}=\frac{1}{n}\sum _{i=1}^n{p}_i.$$

First we prove a proposition that shows, in particular, that if $\widehat{\varphi }(\overline{q})\ne \overline{p}$, then $\varphi =\widehat{\varphi }$ does not minimize

$$\Delta (\varphi )=\sum _{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2,$$

the minimization occurring over either the set of all rigid motions ϕ of ${ℝ}^d$ or the smaller set of rigid motions ϕ of ${ℝ}^d$ that are orientation preserving.

Proposition 2:

Let ϕ be a rigid motion of ${ℝ}^d$ with $\varphi (\overline{q})\ne \overline{p}$ and define an affine linear function τ, $\tau :{ℝ}^d\to {ℝ}^d$, by $\tau (q)=\varphi (q)-\varphi (\overline{q})+\overline{p}\text{for}q\in {ℝ}^d$ for $q\in {ℝ}^d$. Then τ is a rigid motion of ${ℝ}^d,\tau (\overline{q})=\overline{p},\Delta (\tau )<\Delta (\varphi )$, and if ϕ is orientation preserving, then so is τ.

Proof:

Clearly $\tau (\overline{q})=\overline{p}$ Let U be a d × d orthogonal matrix and $t\in {ℝ}^d$ such that ϕ (q) = U q + t for $q\in {ℝ}^d$. Then $\tau (q)=Uq-U\overline{q}+\overline{p}$ so that τ is a rigid motion of ${ℝ}^d$, τ is orientation preserving if ϕ is, and for 1 ≤ i ≤ n, we have

$$\begin{gathered} {\Vert \varphi \left(q_i\right)-p_i\Vert }^2-{\Vert \tau \left(q_i\right)-p_i\Vert }^2={\left(U{q}_i+t-p_i\right)}^T\left(U{q}_i+t-p_i\right)-{\left(U{q}_i-U\overline{q}+\overline{p}-p_i\right)}^T\left(U{q}_i-U\overline{q}+\overline{p}-p_i\right) \\ =\left({\left(U{q}_i-p_i\right)}^T\left(U{q}_i-p_i\right)+2{\left(U{q}_i-p_i\right)}^{T}t+t^{T}t\right)-({\left(U{q}_i-p_i\right)}^T\left(U{q}_i-p_i\right)-2{\left(U{q}_i-p_i\right)}^T\left(U\overline{q}-\overline{p}\right)+{\left(U\overline{q}-\overline{p}\right)}^T\left(U\overline{q}-\overline{p}\right)) \\ =2{\left(U{q}_i-p_i+t\right)}^T\left(U\overline{q}-\overline{p}+t\right)-{\left(U\overline{q}-\overline{p}+t\right)}^T\left(U\overline{q}-\overline{p}+t\right). \end{gathered}$$

It then follows that

$$\begin{gathered} \Delta (\varphi )-\Delta (\tau ) \\ =\sum _{i=1}^n\left(2{\left(U{q}_i-p_i+t\right)}^T(U\overline{q}-\overline{p}+t)-{(U\overline{q}-\overline{p}+t)}^T(U\overline{q}-\overline{p}+t)\right) \\ =n\left|\right|U\overline{q}-\overline{p}+t|{|}^2=n||\varphi (\overline{q})-\overline{p}|{|}^2>0 \end{gathered}$$

as $\varphi (\overline{q})-\overline{p}$ is nonzero. Thus Δ(τ) < Δ(ϕ). □

Finally, the following corollary, a consequence of Proposition 2, shows that the problem of finding an orientation-preserving rigid motion ϕ of ${ℝ}^d$ that minimizes ${\sum }_{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2$ can be reduced to a constrained orthogonal Procrustes problem which, of course, then can be solved with the Kabsch-Umeyama algorithm. Here $r_i=p_i-\overline{p},s_i=q_i-\overline{q}$, for i = 1, …, n, and if $\overline{r}=\frac{1}{n}{\sum }_{i=1}^n{r}_i,\overline{s}=\frac{1}{n}{\sum }_{i=1}^n{s}_i$, then clearly $\overline{r}=\overline{s}=0$.

Corollary 1:

Let $\widehat{U}$ be such that $U=\widehat{U}$ minimizes ${\sum }_{i=1}^n{\Vert U{s}_i-r_i\Vert }^2$ over all d × d rotation matrices U. Let $\widehat{t}=\overline{p}-\widehat{U}\overline{q}$, and let $\widehat{\varphi }$ be given by $\widehat{\varphi }(q)=\widehat{U}q+\widehat{t}$ for $q\in {ℝ}^d$. Then $\varphi =\widehat{\varphi }$ minimizes ${\sum }_{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2$ over all orientation-preserving rigid motions ϕ of ${ℝ}^d$.

Proof:

One such $\widehat{U}$ can be computed with the Kabsch-Umeyama algorithm.

By Proposition 2, in order to minimize ${\sum }_{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2$ over all orientation-preserving rigid motions ϕ of ${ℝ}^d$, it suffices to do it over those for which $\varphi (\overline{q})=\overline{p}$. Therefore, it suffices to minimize ${\sum }_{i=1}^n{\Vert U{q}_i+t-p_i\Vert }^2$ with $t=\overline{p}-U\overline{q}$ over all d × d rotation matrices U, i.e., it suffices to minimize

$$\sum _{i=1}^n{\Vert U{q}_i+\overline{p}-U\overline{q}-p_i\Vert }^2=\sum _{i=1}^n\Vert (U\left(q_i-\overline{q}\right)-\left(p_i-\overline{p}\right){\Vert }^2$$

over all d × d rotation matrices U. But minimizing the last expression is equivalent to minimizing ${\sum }_{i=1}^n{\Vert U{s}_i-r_i\Vert }^2$ over all d × d rotation matrices U. Since $U=\widehat{U}$ is a solution to this last problem, it then follows that $U=\widehat{U}$ minimizes ${\sum }_{i=1}^n{\Vert U{q}_i+\overline{p}-U\overline{q}-p_i\Vert }^2={\sum }_{i=1}^n{\Vert U{q}_i+t-p_i\Vert }^2$ with $t=\overline{p}-U\overline{q}$ over all d × d rotation matrices U. Consequently, if $\widehat{t}=\overline{p}-\widehat{U}\overline{q}$, and $\widehat{\varphi }$ is given by $\widehat{\varphi }(q)=\widehat{U}q+\widehat{t}$ for $q\in {ℝ}^d$, then $\varphi =\widehat{\varphi }$ clearly minimizes ${\sum }_{i=1}^n{\Vert \varphi \left(q_i\right)-p_i\Vert }^2$ over all orientation-preserving rigid motions ϕ of ${ℝ}^d$. □

Biography

About the authors: James F. Lawrence is a professor of mathematics at George Mason University. He is a respected authority in the field of convex and combinatorial geometry. In his position as faculty appointee in the Applied and Computational Mathematics Division at NIST he serves as an expert on combinatorial issues. He received a B.S. in mathematics in 1972 from Oklahoma State University and a Ph.D. in mathematics in 1975 from the University of Washington. He was a National Research Council Postdoctoral Fellow at NIST and held positions at the University of Texas at Austin, the University of Massachusetts at Boston, and the University of Kentucky.

Javier Bernal is a mathematician in the Applied and Computational Mathematics Division of the NIST Information Technology Laboratory in Gaithersburg, MD. He received his Ph.D. in mathematics in 1980 from Catholic University in Washington, DC, the same year he joined NIST. His research interests include the development, analysis and implementation of algorithms in computational geometry, functional and shape data analysis, image processing, optimization, and applied linear algebra.

Christoph Witzgall holds the designation of Scientist Emeritus from the NIST Information Technology Laboratory (ITL). He is a respected authority in the fields of operations research, optimization and numerical analysis. He received his Ph.D. in mathematics in 1958 from the Ludwig-Maximilians-Universität München. Although he retired from government service in 2003, he continues to serve as a guest researcher in the ITL Applied and Computational Mathematics Division. Christoph has been associated with NIST since 1962, serving as Acting Chief of its Operations Research Division from 1979 to 1982. He received the Department of Commerce Silver Medal for meritorious Federal service.

The National Institute of Standards and Technology is an agency of the U.S. Department of Commerce.