Least-Squares Fitting Algorithms of the NIST Algorithm Testing System

Abstract

This report describes algorithms for fitting certain curves and surfaces to points in three dimensions. All fits are based on orthogonal distance regression. The algorithms were developed as reference software for the National Institute of Standards and Technology’s Algorithm Testing System, which has been used for 5 years by NIST and by members of the American Society of Mechanical Engineers’ B89.4.10 standards committee. The Algorithm Testing System itself is described only briefly; the main part of this paper covers the general linear algebra, numerical analysis, and optimization methods it employs. Most of the fitting routines rely on the Levenberg-Marquardt optimization routine.

1. Introduction

Mathematical software, particularly curve and surface fitting routines, is a critical component of coordinate metrology systems. An important but difficult task is to assess the performance of such routines [1,2]. The National Institute of Standards and Technology has developed a software package, the NIST Algorithm Testing System (ATS), that can assist in assessing the performance of geometry-fitting routines [3]. This system has been aiding in the development of a U. S. standard (American Society of Mechanical Engineers (ASME) B89.4.10) for software performance evaluation of coordinate measuring systems [4]. The ATS is also the basis of NIST’s Algorithm Testing and Evaluation Program for Coordinate Measuring Systems (ATEP-CMS) [5,6].

The ATS incorporates three core modules: a data generator [7] for defining and generating test data sets, a collection of reference algorithms which provides a performance baseline for fitting algorithms, and a comparator for analyzing the results of fitting algorithms versus the reference algorithms. This paper concentrates on the development of highly accurate reference algorithms.

The paper is organized as follows. We first introduce notation and certain key functions and derivatives that will be used throughout the paper. Section 2 describes fitting algorithms for linear geometries—planes and lines. Lagrange multipliers [8] are used in solving the constrained minimization problems, which are developed into standard eigenvector problems. Section 3 deals with nonlinear geometry fitting. We use an unconstrained optimization method that requires derivatives of appropriate distance functions. These required functions and derivatives are provided for the reader. Appendix A gives an outline of the unconstrained optimization algorithm (Levenberg-Marquardt) that is modified to allow for normalization of fitting parameters within the routine. Appendix B gives, for all the geometries, the appropriate derivatives needed to create a valuable check for a local minimum.

1.1 Notation and Preliminary Remarks

Assume we are fitting a set of data points, {x_i}, i = 1,2,…, N, that have been translated so that their centroid is the origin. Usually scalar quantities are represented in plain type and matrix or vector quantities with boldface type. Other notation:

x = (x,y,z)	A point in 3-dimensional space.
|·|	The Euclidean (L2) norm. E. g., $|\mathit{x}|=\sqrt{x^2+y^2+z^2}$.
xi = (xi,yi,zi)	The ith data point.
$\overline{\mathit{x}}=(\overline{x},\overline{y},\overline{z})$	The centroid of the data, $\frac{1}{N}\left({\displaystyle \sum x_i,{\displaystyle \sum y_i,{\displaystyle \sum z_i}}}\right)$ (Note: These and all other sums in this paper are taken from i = 1,2,…,N.)
A = (A,B,C)	Direction numbers that specify an orientation, A ≠ 0.
a = (a,b,c)	Direction cosines that specify an orientation. Note: |a| = 1. An orientation’s direction numbers can be converted into direction cosines by: a = A/|A|.
J	The objective function. J is the sum of the squares of the distances from the data points to the geometry. $J={\displaystyle \sum d_i^2}$.
M	The N × 3 matrix containing the data points: $\left[\begin{array}{ccc}{x}_1& y_1& z_1\\ x_2& y_2& z_2\\ ⋮& ⋮& ⋮\\ x_N& y_N& z_N\end{array}\right]$
∇	The gradient of a scalar function. E.g., $\nabla h(x,y,z)=(h_x,h_y,h_z)=\left(\frac{\partial h}{\partial x},\frac{\partial h}{\partial y},\frac{\partial h}{\partial z}\right)$.

For each geometry, we show the defining parameters, the equation for the orthogonal distance from a single data point to the geometry, the objective function, and a brief description of the steps in the calculation.

2. Linear Geometries

Linear geometries (lines and planes) are solved using Lagrange multipliers on a constrained minimization problem. Both cases reduce to a rather simple eigen-problem.

2.1 Plane Fitting

Defining parameters:

• x—a point on the plane.

• a—the direction cosines of the normal to the plane.

Distance equation:

$$d_i=d({\mathit{x}}_i)=d({\mathit{x}}_i,\mathit{x},\mathit{a})=\mathit{a}\cdot ({\mathit{x}}_i-\mathit{x})$$

Objective function:

$$J(\mathit{x},\mathit{a})={{\displaystyle \sum \left[\mathit{a}\cdot ({\mathit{x}}_i-\mathit{x})\right]}}^2$$

Description:

The centroid of the data must lie on the least-squares plane. This can be seen because ∇J = 0 at the least squares solution, yielding ${\displaystyle \sum \mathit{a}\cdot ({\mathit{x}}_i-\mathit{x})=0}$. Multiplying by 1/N gives $\frac{a}{N}{\displaystyle \sum (x_i-x)+\frac{b}{N}{\displaystyle \sum (y_i-y)}}+\frac{c}{N}{\displaystyle \sum (z_i-z)}=0$. Distributing the summation gives $a(\overline{x}-x)+b(\overline{y}-y)+c(\overline{z}-z)=0$, which is to say $d(\overline{\mathit{x}},\mathit{x},\mathit{A})=0$, i.e., $\overline{\mathit{x}}$ lies on the least-squares plane. Since by assumption the data points have been translated to the origin, and since the centroid of the data must be a point on the least squares plane, we can set x = 0.

The direction of the fitted plane, a, can be found by solving the constrained minimization problem, namely, minimizing J subject to the constraint that |a| = 1. Define a function, G = |a|² − 1, so that the problem is to minimize J subject to the constraint that G = 0. The method of Lagrange multipliers [8] tells us that the minimum occurs at a point where ∇J = λ∇G, for some real number λ. (Here, a, b, and c are treated as independent variables, since the constraint is accounted for in G. Therefore, ∇ = (∂/∂a, ∂/∂b, ∂/∂c).) But ∇G = 2a, and ∇J = 2(M^⊤M)a, yielding the eigen-problem, (M^⊤M)a = λa, referred to as the normal equations.

This 3 × 3 eigenvector problem can be easily solved using well-established routines (e.g., Jacobi iterations [9]). However, we note that the eigenvectors of M^⊤M are also the singular vectors (from the singular value decomposition) of M [9]. This allows us to gain numerical stability by applying the singular value decomposition (SVD) to M without ever computing M^⊤M, which is the method implemented in the ATS.

Finally, we must determine how to select the correct eigenvector (i.e., singular vector) of the three produced by the SVD. The normal equations can be written as follows:

$$\begin{array}{l}\mathrm{\Sigma }{x}_i(\mathit{a}\cdot {\mathit{x}}_i)=\mathrm{\lambda }a\\ \mathrm{\Sigma }{y}_i(\mathit{a}\cdot {\mathit{x}}_i)=\mathrm{\lambda }b\\ \mathrm{\Sigma }{z}_i(\mathit{a}\cdot {\mathit{x}}_i)=\mathrm{\lambda }c\end{array}$$

Multiplying these three equations by a, b, and c, respectively, then summing the equations gives ${\displaystyle \sum {\left(\mathit{a}\cdot {\mathit{x}}_i\right)}^2=\lambda {|\mathit{a}|}^2=}\lambda$.

But the sum on the left is just the objective function, i.e., the sum of the squares of the residuals. Thus the value of λ is the objective function, hence the correct eigenvector for the least-squares solution corresponds to the smallest eigenvalue. When using the SVD, we choose the singular vector corresponding to the smallest singular value, since under these conditions the singular values are the square roots of the eigenvalues [10].

2.2 Line Fitting

Defining parameters:

• x—a point on the line.

• a—the direction cosines of the line.

Distance equation:

d(xi) = d(xi, x,a) = |a × (xi − x)| (which also equals $\sqrt{{|{\mathit{x}}_i-\mathit{x}|}^2-{\left[\mathit{a}\cdot ({\mathit{x}}_i-\mathit{x})\right]}^2}$ where × denotes the vector cross product.

Objective function:

$$J(\mathit{x},\mathit{a})={\displaystyle \sum {|\mathit{a}\times ({\mathit{x}}_i-\mathit{x})|}^2}$$

Description:

Since the data has been translated to the origin, and since the centroid of the data must be a point on the least squares plane (similar to the case above with the least squares plane), we set x = 0.

The direction a can be found by following the same strategy as in the case of a plane. For line fitting, the normal equations are M^⊤Ma = λa just as in the case of plane fitting. Once again, the correct eigenvector must be chosen to minimize the sum-of-squares of the residuals. As shown with planes, we obtain λ = Σ(a·x)², so J = − λ + ∑|x_i|², meaning that J is minimized when λ is maximized. Thus the correct eigenvector choice is the one corresponding to the largest eigenvalue. As in the case of plane fitting, numerical stability is gained by finding the eigenvectors of M through the SVD, rather than by solving the normal equations. Since the singular values are the square roots of the eigenvalues [10], we choose the eigenvector corresponding to the largest singular value.

3. Nonlinear Geometries

3.1 Utility Functions f and g

The line and plane distance functions arise quite often in this paper, thus we define them here, calling them f and g respectively, giving necessary derivatives, which are used throughout the rest of this paper. We compute the nonlinear fits using unconstrained minimization algorithms, so we define the line and plane distance functions in terms of direction numbers rather than direction cosines.

Let g(x_i,x,A) denote the distance from the point, x_i, to the plane defined by the point, x, and the normal direction, a = A/|A|. The value of g is given by: g_i = g(x_i,x,A) = a·(x_i − x) = a(x_i − x) + b(y_i − y) + c(z_i − z).

Let f(x_i,x,A) denote the distance from the point, x_i, to the line defined by the point, x, and the direction, a = A/|A|. The value of f is given by: f_i = f(x_i,x,A) = |a × (x_i − x)|. That is,

$$\begin{array}{r}u=c(y_i-y)-b(z_i-z)\\ f_i=\sqrt{u^2+v^2+w^2},\text{where}v=a(z_i-z)-c(x_i-x)\\ w=b(x_i-x)-a(y_i-y)\end{array}$$

This expression for f is used because of its numerical stability. One should note that f could also be expressed (for derivative calculations) as $f_i=\sqrt{{|{\mathit{x}}_i-\mathit{x}|}^2-g_i^2}$.

Note: A, B, and C are independent variables, whereas a, b, and c are not, because the constraint a² + b² + c² = 1 causes a, b, and c to depend on each other. When dealing with the nonlinear geometries we treat f and g as functions dependent on A, as opposed to a, in order to use unconstrained minimization algorithms. Treating f and g as functions dependent on a would force us to restrict ourselves to using constrained minimization solvers. In the linear cases, we did solve constrained minimization problems. So when we differentiate with respect to A, for example, we treat a, b, and c all as functions of A, B, and C (e.g., $a=A/\sqrt{A^2+B^2+C^2})$. This yields the following array of derivatives:

$$\left[\begin{array}{c}\nabla a\\ \nabla b\\ \nabla c\end{array}\right]=\left[\begin{array}{ccc}\frac{\partial a}{\partial A}& \frac{\partial a}{\partial B}& \frac{\partial a}{\partial C}\\ \frac{\partial b}{\partial A}& \frac{\partial b}{\partial B}& \frac{\partial b}{\partial C}\\ \frac{\partial c}{\partial A}& \frac{\partial c}{\partial B}& \frac{\partial c}{\partial C}\end{array}\right]=\frac{1}{|\mathit{A}|}\left[\begin{array}{lll}1-a^2& -ab& -ac\\ -ab& 1-b^2& -bc\\ -ac& -bc& 1-c^2\end{array}\right]$$

These algorithms normalize A at every step, so for simplicity of expressing derivatives, assume |A| = 1. A remains unconstrained; we just assume it happens to have unit magnitude.

The derivatives for f_i and g_i are then

$$\begin{array}{l}\frac{\partial g_i}{\partial x}=-a\\ \frac{\partial g_i}{\partial y}=-b\\ \frac{\partial g_i}{\partial z}=-c\\ \frac{\partial g_i}{\partial A}=(x_i-x)-a{g}_i\\ \frac{\partial g_i}{\partial B}=(y_i-y)-b{g}_i\\ \frac{\partial g_i}{\partial C}=(z_i-z)-c{g}_i\\ \frac{\partial f_i}{\partial x}=[a{g}_i-(x_i-x)]/f_i\\ \frac{\partial f_i}{\partial y}=[b{g}_i-(y_i-y)]/f_i\\ \frac{\partial f_i}{\partial z}=[c{g}_i-(z_i-z)]/f_i\\ \frac{\partial f_i}{\partial A}=g_i[a{g}_i-(x_i-x)]/f_i\\ \frac{\partial f_i}{\partial B}=g_i[b{g}_i-(y_i-y)]/f_i\\ \frac{\partial f_i}{\partial C}=g_i[c{g}_i-(z_i-z)]/f_i\end{array}$$

The above derivatives of f_i are undefined when f_i = 0 (i.e., when x_i is on the line.) In this rarely needed case the gradient is given by:

$$\begin{array}{l}(\sqrt{1-a^2},\sqrt{1-b^2},\sqrt{1-c^2},\\ g\sqrt{1-a^2},g\sqrt{1-b^2},g\sqrt{1-c^2}).\end{array}$$

For cylinders, cones, and tori, the line associated with f is the geometry’s axis. For cones the plane associated with g is the plane through the point, x, perpendicular to the axis. For tori, the plane associated with g is the plane perpendicular to the axis that divides the torus in half.

3.2 Choice of Algorithm

Good optimization algorithms can readily be found to minimize the objective function, J. Usually such an algorithm will require an initial guess, along with partial derivatives, either of J itself or of the distance function, d_i. Both sets of derivatives are given in this paper (those for J in the appendix). These should enable a reader to implement a least-squares algorithm even if the optimization algorithm used differs from the author’s choice, which follows.

In the ATS, nonlinear geometries are fit in an unconstrained manner using the Levenberg-Marquardt algorithm. The algorithm requires an initial guess as well as the first derivatives of the distance function. In practice it converges quickly and accurately even with a wide range of initial guesses. Details of this algorithm are given in appendix A. Additionally, the code allows us to normalize the fitting parameters after every iteration of the algorithm. For each geometry we list the distance and objective functions, the appropriate derivatives, and the parameter normalization we use.

3.3 Sphere Fitting

Defining parameters:

• x—the center of the sphere.

• r—the radius of the sphere.

Distance equation:

$$d({\mathit{x}}_i)=|{\mathit{x}}_i-\mathit{x}|-r$$

Objective function:

$$J({\mathit{x}}_{i}r)={\displaystyle \sum {\left(|{\mathit{x}}_i-\mathit{x}|-r\right)}^2}$$

Normalization:

(None)

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}=-(x_i-x)/|{\mathit{x}}_i-\mathit{x}|\\ \frac{\partial d_i}{\partial y}=-(y_i-y)/|{\mathit{x}}_i-\mathit{x}|\\ \frac{\partial d_i}{\partial z}=-(z_i-z)/|{\mathit{x}}_i-\mathit{x}|\\ \frac{\partial d_i}{\partial r}=-1\end{array}$$

3.4 Two-Dimensional Circle Fitting

This case is simply the sphere fit (above) restricted to two-dimensions:

Defining parameters:

• x—the x-coordinate of the center of the circle.

• y—the y-coordinate of the center of the circle.

• r—the radius of the circle.

Distance equation:

$$d(x_i,y_i)=\sqrt{{\left(x_i-x\right)}^2+{\left(y_i-y\right)}^2}-r$$

Objective function:

$$J(x,y,r)={\displaystyle \sum {\left(\sqrt{{\left(x_i-x\right)}^2+{\left(y_i-y\right)}^2}-r\right)}^2}$$

Normalization:

(None)

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}=-(x_i-x)/(d_i+r)\\ \frac{\partial d_i}{\partial y}=-(y_i-y)/(d_i+r)\\ \frac{\partial d_i}{\partial r}=-1\end{array}$$

3.5 Three-Dimensional Circle Fitting

Defining parameters:

• x—the center of the circle.

• A—the direction numbers of the normal to circle’s plane.

• r—the radius of the circle.

Distance equation:

$$d({\mathit{x}}_i)=\sqrt{g_i^2+{\left(f_i-r\right)}^2}$$

Objective function:

$$J(\mathit{x},\mathit{A},r)={\displaystyle \sum \left(g_i^2+{\left(f_i-r\right)}^2\right)}$$

Normalization:

A ← A/|A|

(Here and elsewhere, “←” denotes assignment of value. In this case, the value of A is replaced by the value A/|A|.)

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}=\left[g_i{\left(g_i\right)}_x+f_i{\left(f_i\right)}_x\right]/d_i\\ \frac{\partial d_i}{\partial y}=\left[g_i{\left(g_i\right)}_y+f_i{\left(f_i\right)}_y\right]/d_i\\ \frac{\partial d_i}{\partial z}=\left[g_i{\left(g_i\right)}_z+f_i{\left(f_i\right)}_z\right]/d_i\\ \frac{\partial d_i}{\partial A}=\left[g_i{\left(g_i\right)}_A+f_i{\left(f_i\right)}_A\right]/d_i\\ \frac{\partial d_i}{\partial B}=\left[g_i{\left(g_i\right)}_B+f_i{\left(f_i\right)}_B\right]/d_i\\ \frac{\partial d_i}{\partial C}=\left[g_i{\left(g_i\right)}_C+f_i{\left(f_i\right)}_C\right]/d_i\\ \frac{\partial d_i}{\partial r}=-\left(f_i-r\right)/d_i\end{array}$$

Description:

We use a multi-step process to accomplish 3D circle fitting:

1. Compute the least-squares plane of the data.

2. Rotate the data such that the least-squares plane is the x-y plane.

3. Project the rotated data points onto the x-y plane.

4. Compute the 2D circle fit in the x-y plane.

5. Rotate back to the original orientation.

6. Perform a full 3D minimization search over all the parameters.

Some coordinate measuring system software packages stop at step (5) and report the orientation, center, and radius as the least-squares circle in 3D. This approach is valid when the projection onto the plane is done simply to compensate for measurement errors on points which would otherwise be coplanar. But this method does not in general produce the circle yielding the least sum-of-squares possible (even though it is usually a good approximation.) In order to achieve the true 3D least-squares fit, the circle computed at step (5) is used as an initial guess in the Levenberg-Marquardt algorithm, which optimizes over all the parameters simultaneously [step (6)].

Step (2) is carried out using the appropriate rotation matrix to rotate the direction, a, to the z-direction, namely,

$$\left[\begin{array}{ccc}1-\frac{a^2}{1+c}& \frac{-ab}{1+c}& -a\\ \frac{-ab}{1+c}& 1-\frac{b^2}{1+c}& -b\\ a& b& c\end{array}\right]$$

If c < 0, a is replaced with − a, thus rotating the direction to the minus z-direction (which is adequate for our purposes.) Step (5) is carried out using the appropriate rotation matrix to rotate the z-direction to the direction, a. Namely,

$$\left[\begin{array}{ccc}1-\frac{a^2}{1+c}& \frac{-ab}{1+c}& a\\ \frac{-ab}{1+c}& 1-\frac{b^2}{1+c}& b\\ -a& -b& c\end{array}\right]$$

3.6 Cylinder Fitting

Defining parameters:

• x—a point on the cylinder axis.

• A—the direction numbers of the cylinder axis.

• r—the radius of the cylinder.

Distance equation:

$$d({\mathit{x}}_i)=f_i-r$$

Objective function:

$$J(\mathit{x},A,r)={\displaystyle \sum {\left(f_i-r\right)}^2}$$

Normalization:

• A ← A/|A|

• X ← (point on axis closest to origin)

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}={\left(f_i\right)}_x\\ \frac{\partial d_i}{\partial y}={\left(f_i\right)}_y\\ \frac{\partial d_i}{\partial z}={\left(f_i\right)}_z\\ \frac{\partial d_i}{\partial A}={\left(f_i\right)}_A\\ \frac{\partial d_i}{\partial B}={\left(f_i\right)}_B\\ \frac{\partial d_i}{\partial C}={\left(f_i\right)}_C\\ \frac{\partial d_i}{\partial r}=-1\end{array}$$

3.7 Cone Fitting

Defining parameters:

• x—a point on the cone axis (not the apex).

• A—the direction numbers of the cone axis (pointing toward the apex).

• s—the orthogonal distance from the point, x, to the cone.

• ψ —the cone’s apex semi-angle.

Distance equation:

$$d({\mathit{x}}_i)=f_i\cos \psi +g_i\sin \psi -s$$

Objective function:

$$J(\mathit{x},\mathit{A},s,\psi )={\displaystyle \sum {\left(f_i\cos \psi +g_i\sin \psi -s\right)}^2}$$

Normalization:

• A ← A/|A|

• x ← (point on axis closest to origin)

• ψ ← ψ (mod 2π)

• if ψ > π then [ψ ← ψ (mod π); A ← − A]

• if $\psi >\frac{\pi }{2}$ then ψ ← π – ψ

• if s < 0 then [s ← − s; A ← − A]

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}={\left(f_i\right)}_x\cos \psi +{\left(g_i\right)}_x\sin \psi \\ \frac{\partial d_i}{\partial y}={\left(f_i\right)}_y\cos \psi +{\left(g_i\right)}_y\sin \psi \\ \frac{\partial d_i}{\partial z}={\left(f_i\right)}_z\cos \psi +{\left(g_i\right)}_z\sin \psi \\ \frac{\partial d_i}{\partial A}={\left(f_i\right)}_A\cos \psi +{\left(g_i\right)}_A\sin \psi \\ \frac{\partial d_i}{\partial B}={\left(f_i\right)}_x\cos \psi +{\left(g_i\right)}_x\sin \psi \\ \frac{\partial d_i}{\partial C}={\left(f_i\right)}_x\cos \psi +{\left(g_i\right)}_x\sin \psi \\ \frac{\partial d_i}{\partial s}=-1\\ \frac{\partial d_i}{\partial \psi }=-f_i\sin \psi +g_i\cos \psi \end{array}$$

3.8 Torus Fitting

Defining parameters:

• x—the torus center.

• A—the direction numbers of the torus axis.

• r—the major radius.

• R—the minor radius.

Distance equation:

$$d({\mathit{x}}_i)=\sqrt{g_i^2+{\left(f_i-r\right)}^2-R}$$

Objective function:

$$J(\mathit{x},\mathit{A},r,R)={\displaystyle \sum {\left[\sqrt{g_i^2+{\left(f_i-r\right)}^2}-R\right]}^2}$$

Normalization:

• A ← A/|A|

Derivatives:

$$\begin{array}{l}\frac{\partial d_i}{\partial x}=\left[g_i{\left(g_i\right)}_x+\left(f_i-r\right){\left(f_i\right)}_x\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial y}=\left[g_i{\left(g_i\right)}_y+\left(f_i-r\right){\left(f_i\right)}_y\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial z}=\left[g_i{\left(g_i\right)}_z+\left(f_i-r\right){\left(f_i\right)}_z\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial A}=\left[g_i{\left(g_i\right)}_A+\left(f_i-r\right){\left(f_i\right)}_A\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial B}=\left[g_i{\left(g_i\right)}_B+\left(f_i-r\right){\left(f_i\right)}_B\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial C}=\left[g_i{\left(g_i\right)}_C+\left(f_i-r\right){\left(f_i\right)}_C\right]/\left(d_i+R\right)\\ \frac{\partial d_i}{\partial r}=-(f_i-r)(d_i+R)\\ \frac{\partial d_i}{\partial R}=-1\end{array}$$

4. Discussion

The algorithms have been implemented in the ATS and have been used for 5 years by NIST and by members of the ASME B89.4.10 Working Group. In general they have performed extremely well. They have successfully solved a number of difficult fitting problems that could not be solved by many commercial software packages used on Coordinate Measuring Systems (CMSs). (Some of the most difficult problems are cylinders or cones sampled over a small patch.) The ATS algorithms have an extremely broad range of convergence. Failure to converge has only been observed for pathological fitting problems (e.g., fitting a circle to collinear points). Special checks can detect most of these situations.

The ATS algorithms are generally robust. For most fits, a good starting guess is not required to reach the global minimum. This is due, in part, to the careful choice of fitting parameters, the use of certain constraints, and, for cylinders and cones, the technique of restarting a search after an initial solution is found.

Fig. 1.

Levenberg-Marquardt algorithm.

Biography

About the author: Craig Shakarji is a mathematician in the Manufacturing Systems Integration Division of the NIST Manufacturing Engineering Laboratory. The National Institute of Standards and Technology is an agency of the Technology Administration, U.S. Department of Commerce.

5. Appendix A. The Levenberg-Marquardt Algorithm

The Levenberg-Marquardt algorithm [11] finds the vector, p, that minimizes an objective function, $J(\mathit{p})=\Sigma d_i^2(\mathit{p})$, where d_i(p) is the distance from the point, x_i, to the geometry defined by the vector of parameters, p. In the case of a cylinder, for example, p = (x, A, r) = (x, y, z, A, B, C, r) and d_i(p) = f_i − r. One derivation of the algorithm [12] begins by approximating J as a linear function of p, $\widehat{J}(\mathit{p})$:

$$J(\mathit{p})\approx \widehat{J}(\mathit{p})={\displaystyle \sum {\left(d_i({\mathit{p}}_0)+\nabla d_i({\mathit{p}}_0)\cdot \mathit{p}\right)}^2}$$

where ∇d_i(p₀) is the gradient of d_i(p) evaluated at an initial guess, p₀. This approximation is valid within a certain trust region radius. The derivation then considers how to minimize $\widehat{J}(\mathit{p})$. A search direction is calculated based on $\widehat{J}(\mathit{p})$, and a search is made in that direction within the limits of the trust region radius for a point, p_new, such that J(p_new) < J(p₀). When p_new is found, it becomes the new p₀ for another iteration of the above process.

The basis for the algorithm is the result that the solution, p*, to each iteration can be expressed as p* = p(λ) = − (F₀^⊤F₀ + λD^⊤D)⁻¹ F₀^⊤d(p₀) where F₀ is a matrix having ∇d_i(p₀) as its ith row, D is an appropriate weighting matrix, d(p₀) is the vector of residuals, d_i(p₀), and λ ≥ 0 is a variable called the Levenberg-Marquardt parameter. This parameter can be considered the Lagrange multiplier for the constraint that each search be limited to the trust region radius.

The Levenberg-Marquardt algorithm is presented in the figure below. The matrix F₀^⊤F₀ + λD^⊤D is named H and the vector F₀^⊤d(p₀) is called v. The algorithm includes a modification suggested by Nash [11], which is to use a weighting matrix defined so that D^⊤D is the identity matrix plus the diagonal of F₀^⊤F₀. Nash’s use of the identity in the definition of D forces H to be positive definite. (Note that D is never calculated.) Since H is also symmetric, the system Hx = − v can be reliably solved using the Cholesky decomposition [9]. We chose 10 and 0.04 as factors with which to increment and decrement λ respectively. Finally, note that we normalize the parameter vector, p, at every iteration. However, normalization of p never changes the value of the objective function, J(p). This routine is the basis of the ATS implementation of the Levenberg-Marquardt algorithm.

6. Appendix B. Gradient Test for Correctness

One helpful check is to compute the gradient of the objective function at the computed minimum. The correct least-squares solution yields ΔJ = 0. Also, these derivatives are important because several minimization algorithms require that these derivatives be included. The definitions of f and g are kept the same as they were in Sec. 3. For every nonlinear geometry, assume the same definition for the objective function, J, as specified in Sec. 3. For the linear geometries, the objective functions are here defined in terms of unconstrained parameters.

Planes

$J(\mathit{x},\mathit{A})={\displaystyle \sum g_i^2}$

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum g_i{\left(g_i\right)}_x}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum g_i{\left(g_i\right)}_y}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum g_i{\left(g_i\right)}_z}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum g_i{\left(g_i\right)}_A}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum g_i{\left(g_i\right)}_B}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum g_i{\left(g_i\right)}_C}\end{array}$

Lines

$J(\mathit{x},\mathit{A})={\displaystyle \sum f_i^2}$

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum f_i{\left(f_i\right)}_x}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum f_i{\left(f_i\right)}_y}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum f_i{\left(f_i\right)}_z}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum f_i{\left(f_i\right)}_A}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum f_i{\left(f_i\right)}_B}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum f_i{\left(f_i\right)}_C}\end{array}$

Spheres

$\begin{array}{l}\frac{\partial J}{\partial x}=-2{\displaystyle \sum d_i(x_i-x)/|{\mathit{x}}_i-\mathit{x}|}\\ \frac{\partial J}{\partial y}=-2{\displaystyle \sum d_i(y_i-y)/|{\mathit{x}}_i-\mathit{x}|}\\ \frac{\partial J}{\partial z}=-2{\displaystyle \sum d_i(z_i-z)/|{\mathit{x}}_i-\mathit{x}|}\\ \frac{\partial J}{\partial r}=-2{\displaystyle \sum d_i}\end{array}$

2D Circles

$\begin{array}{l}\frac{\partial J}{\partial x}=-2{\displaystyle \sum d_i(x_i-x)/(d_i+r)}\\ \frac{\partial J}{\partial y}=-2{\displaystyle \sum d_i(y_i-y)/(d_i+r)}\\ \frac{\partial J}{\partial r}=-2{\displaystyle \sum d_i}\end{array}$

3D Circles

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_x+\left(f_i-r\right){\left(f_i\right)}_x\right]}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_y+\left(f_i-r\right){\left(f_i\right)}_y\right]}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_z+\left(f_i-r\right){\left(f_i\right)}_z\right]}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_A+\left(f_i-r\right){\left(f_i\right)}_A\right]}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_B+\left(f_i-r\right){\left(f_i\right)}_B\right]}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum \left[g_i{\left(g_i\right)}_C+\left(f_i-r\right){\left(f_i\right)}_C\right]}\\ \frac{\partial J}{\partial r}=-2{\displaystyle \sum \left(f_i-r\right)}\end{array}$

Cylinders

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_x}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_y}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_z}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_A}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_B}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum \left(f_i-r\right){\left(f_i\right)}_C}\\ \frac{\partial J}{\partial r}=-2{\displaystyle \sum \left(f_i-r\right)}\end{array}$

Cones

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_x\cos \psi +{\left(g_i\right)}_x\sin \psi \right]}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_y\cos \psi +{\left(g_i\right)}_y\sin \psi \right]}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_z\cos \psi +{\left(g_i\right)}_z\sin \psi \right]}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_A\cos \psi +{\left(g_i\right)}_A\sin \psi \right]}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_B\cos \psi +{\left(g_i\right)}_B\sin \psi \right]}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum d_i\left[{\left(f_i\right)}_C\cos \psi +{\left(g_i\right)}_C\sin \psi \right]}\\ \frac{\partial J}{\partial \psi }=2{\displaystyle \sum d_i\left[-f_i\sin \psi +g_i\cos \psi \right]}\\ \frac{\partial J}{\partial s}=-2{\displaystyle \sum d_i}\end{array}$

Tori

$\begin{array}{l}\frac{\partial J}{\partial x}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_x+\left(f_i-r\right){\left(f_i\right)}_x\right]}\\ \frac{\partial J}{\partial y}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_y+\left(f_i-r\right){\left(f_i\right)}_y\right]}\\ \frac{\partial J}{\partial z}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_z+\left(f_i-r\right){\left(f_i\right)}_z\right]}\\ \frac{\partial J}{\partial A}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_A+\left(f_i-r\right){\left(f_i\right)}_A\right]}\\ \frac{\partial J}{\partial B}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_B+\left(f_i-r\right){\left(f_i\right)}_B\right]}\\ \frac{\partial J}{\partial C}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_C+\left(f_i-r\right){\left(f_i\right)}_C\right]}\\ \frac{\partial J}{\partial r}=2{\displaystyle \sum \frac{d_i}{d_i+R}\left[g_i{\left(g_i\right)}_r-\left(f_i-r\right)\right]}\\ \frac{\partial J}{\partial R}=-2{\displaystyle \sum d_i}\end{array}$