Calibration of mobile manipulators using 2D positional features

Abstract

Robotic manipulators are increasingly being attached to Automatic Ground Vehicles (AGVs) to aid in the efficiency of assembly for manufacturing systems. However, calibrating these mobile manipulators is difficult as the offset between the robotic manipulator and the AGV is often unknown. This paper provides a novel, simple, and low-cost method for calibrating and measuring the performance of mobile manipulators by using data collected from a laser retroreflector that digitally detects the horizontal two-dimensional (2D) position of reflectors on an artifact as well as a navigation system that provides the heading angle and 2D position of the AGV. The method is mathematically presented by providing a closed form solution to the positional component of the 2D robotworld/hand-eye calibration problem AX Y= B. The method is then applied to simulated data as well as data collected in a laboratory setting and compared to other methods.

1. Introduction

Automatic guided vehicles (AGVs) have traditionally been used to transport objects between workstations containing stationary robotic manipulators. However, difficulties arise when these objects are not accessible by the AGVs. For these situations, it may be beneficial to attach a robotic manipulator onto an AGV to allow for much greater capability in object access by the AGV without requiring fixed infrastructure (e.g., conveyers). These “mobile manipulators” may also have the ability to replace multiple stationary manipulators which can result in actual cost savings for the manufacturer. However, the use of mobile manipulators may present some challenges for the manufacturer. For instance, mobile manipulators often use intelligent sensing systems, such as vision [1], that measure the pose of a given object. These measurements are calculated with respect to the system’s own coordinate frame. Therefore, techniques are needed to transform the measurements into a common world coordinate frame.

Consider the setup shown in Fig. 1 where a robotic manipulator is attached to an AGV. A simple and low-cost method for calibrating and measuring the performance of this mobile manipulator is formulated by wielding a laser retroreflector to the robotic manipulator as an end-ofarm-tool. The retroreflector has a binary output which turns on when it detects a reflector (target) that is attached to a Reconfigurable Mobile Manipulator Artifact (RMMA) [2,3]. The horizontal two-dimensional (2D) position of the robot arm with respect to its base is recorded when a reflector is detected. Here the reflector diameters can be selected to be 1 mm or larger based on the expected uncertainty of the mobile manipulator to align with pre-taught locations on the RMMA. The distances between reflectors are within machining tolerances and the uncertainty in position is ±0.25 mm. At the same time, a navigation system computes the 2D (x,y) position and heading angle (θ) of the AGV in world coordinates with a rotating laser range sensor that detects facility-based reflectors mounted throughout the mobile manipulator work area.

Fig. 1.

Experimental setup: A robotic manipulator is attached to an AGV to create a mobile manipulator. The goals of this paper are to (1) calculate the offset between the robotic manipulator and the AGV and (2) compute the target (reflectors used for registration and assembly performance measurement) locations with respect to a fixed world coordinate system. Follow the arrows (right) from the AGV to see A_iX = YB_i. Here the transformation Y from the AGV to the robotic manipulator is unknown as well as the transformation X from the world to the targets.

The mobile manipulator is programmed to reach different locations around the workroom and searches for predefined target locations. When a target is located, the x y, θ of the AGV is recorded as well as the x,y position of the robotic manipulator (see Table 5 in Appendix A). Since the robotic manipulator is fixed to the non-stationary AGV, its coordinate frame moves as the AGV moves. Therefore, each target location is with respect to a moving coordinate frame. To allow for a complete theoretical understanding of the mobile manipulator calibration process as well as to provide a performance measurement for the mobile manipulator, the target locations with respect to a stationary world coordinate frame is needed. To compute these locations, the transformation from the AGV to the robotic manipulator must be found which is a goal of this paper.

These types of problems are often represented as a robot-world/ hand-eye calibration problem and are mathematically formulated as

$${\mathbf{A}}_i\mathbf{X}=\mathbf{Y}{\mathbf{B}}_i,$$

where A_i and B_i are homogeneous transformation matrices representing data that are measured at time-step i = 1,2, … n, while X and Y are the unknown homogeneous transformation matrices (see Fig. 1). It should be noted that this setup can also be used for other problems such as calibrating a robot base frame with the world frame [4]. For this paper

A_i=inverse of the AGV pose in world coordinates

B_i=target pose in robotic manipulator coordinates

X=target pose in world coordinates

Y=transformation from AGV to robotic manipulator.

Here homogeneous transformation matrices are of the form $\left(\begin{array}{ll}\mathbf{R}\hfill & \mathbf{t}\hfill \\ 0\hfill & 1\hfill \end{array}\right)$ where orientation is represented by the rotation matrix R and position is represented as the vector t. Using this representation, the robotworld/hand-eye calibration problem can be posed as

$$\left(\begin{array}{cc}{\mathbf{R}}_{{\mathbf{A}}_i}& {\mathbf{t}}_{{\mathbf{A}}_i}\\ 0& 1\end{array}\right)\left(\begin{array}{cc}{\mathbf{R}}_{\mathbf{X}}& {\mathbf{t}}_{\mathbf{X}}\\ 0& 1\end{array}\right)=\left(\begin{array}{cc}{\mathbf{R}}_{\mathbf{Y}}& {\mathbf{t}}_{\mathbf{Y}}\\ 0& 1\end{array}\right)\left(\begin{array}{cc}{\mathbf{R}}_{{\mathbf{B}}_i}& {\mathbf{t}}_{{\mathbf{B}}_i}\\ 0& 1\end{array}\right),$$

which can be split into its orientational component

$${\mathbf{R}}_{{\mathbf{A}}_i}{\mathbf{R}}_{\mathbf{X}}={\mathbf{R}}_{\mathbf{Y}}{\mathbf{R}}_{{\mathbf{B}}_i}$$ (1)

and positional component

$${\mathbf{R}}_{{\mathbf{A}}_i}{\mathbf{t}}_{\mathbf{X}}+{\mathbf{t}}_{{\mathbf{A}}_i}={\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_i}+{\mathbf{t}}_{\mathbf{Y}}.$$ (2)

Solutions to these problems come in two types: iterative and closed form. Iterative solutions [5–11] are generally very accurate, but their solutions are based on initial guesses and can be slow. In contrast, closed form solutions are fast and can be used as initial guesses to iterative solutions. This paper will focus on closed form solutions. Closed form solutions come in two types: simultaneous solutions which solve for the orientational and positional components at the same time and separable solutions which separately solve for the orientational component and positional component [12,8]. There have been many closed form solutions to AX = YB which include using quaternions [13,6], non-orthogonal matrices [14], probability density functions [15], and Kronecker products [4,16,17]. Many of these methods are based on solutions to the simplified AX = XB problem [18–30] and have been generalized to the multi-robot calibration problem AXB = YCZ [31–33]. Due to the sensors being used, the orientational information ${\mathbf{R}}_{{\mathbf{B}}_i}$ are not computed, the orientational information ${\mathbf{R}}_{{\mathbf{A}}_i}=\left(\begin{array}{cc}\cos {\theta }_i& \sin {\theta }_i\\ -\sin {\theta }_i& \cos {\theta }_i\end{array}\right)$ consist of only the heading angles θ_i, and the positional information ${\mathbf{t}}_{{\mathbf{A}}_i},{\mathbf{t}}_{{\mathbf{B}}_i}$ are only in two-dimensions. Therefore, most closed form solutions are not applicable as the orientational information ${\mathbf{R}}_{{\mathbf{B}}_i}$ found in the orientational component of Eq. (1) are not available. However, the methods presented in [14,4] solve for the positional component without the need of the orientational component. For both of these papers, the unknowns are solved as minimization problems without the additional constraint of orthogonalization of the unknown rotation matrix R_Y. This orthogonalization is later applied as an additional step. The closed form solution presented in this paper builds upon this work but assumes the orthogonality of R_Y in its original formulation. In addition, the closed form solution presented in this paper is generalized to the case where the location of more than one target location t_X in world coordinates is needed. The work presented in this paper extends existing calibration tools [34–36] that may be useful for robotic systems.

It should be noted that this paper focuses on just solving the positional component of Eq. (2). This is sufficient for the setup of this paper as just the target locations in world coordinates (t_X) and the sensor calibration (R_Y t_Y ) are needed. However, if the orientational information ${\mathbf{R}}_{{\mathbf{B}}_i}$ are provided then the full solution (including R_X) of the robotworld/hand-eye calibration problem could be found by solving the orientational component of Eq. (1) as an orthogonal Procrustes problem. Motivation for the work presented in this paper comes from [37] where a similar technique is applied to the exterior orientation problem and from the classic Wang and Jepson method for solving the absolute orientation problem [38].

This paper is organized as followed: Section 2 will give the methodology for solving the positional component shown in Eq. (2) in closed form for a single target. Then the methodology is extended for multiple targets. Section 3 will describe experiments comparing the closed form solution with the Wu and Ren closed form solution [4] and the iterative solution presented in [39], and Section 4 will give concluding remarks. For this paper, ‖·‖ denotes the Frobenius norm, so

$$\Vert \mathbf{A}\Vert =\sqrt{\mathrm{tr}\left(\mathbf{A}{\mathbf{A}}^T\right)}=\sqrt{\mathrm{tr}\left({\mathbf{A}}^T\mathbf{A}\right)}$$

where ^T denotes the transpose operator and tr(A) is the trace of the matrix A.

2. Methods

2.1. Single target

The goal of this paper is to determine the location t_X of a given target with respect to the world coordinate via the computation of the transformation $\mathbf{Y}=\left(\begin{array}{cc}{\mathbf{R}}_{\mathbf{Y}}& {\mathbf{t}}_{\mathbf{Y}}\\ 0& 1\end{array}\right)$ between the AGV and robotic manip-ulator. Mathematically, the equation that formulates this relationship is the positional component, shown in Eq. (2), of the robot-world/handeye calibration problem. Assuming that the data gathered is noiseless, then each side of this equation can be multiplied by scalars c_i such that

$$c_i{\mathbf{R}}_{{\mathbf{A}}_i}{\mathbf{t}}_{\mathbf{X}}+c_i{\mathbf{t}}_{{\mathbf{A}}_i}=c_i{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_i}+c_i{\mathbf{t}}_{\mathbf{Y}}$$

for all i = 1,2,…,n. As a result,

$${\displaystyle \sum _{i=1}^n\left(c_i{\mathbf{R}}_{{\mathbf{A}}_i}{\mathbf{t}}_{\mathbf{X}}+c_i{\mathbf{t}}_{{\mathbf{A}}_i}\right)}={\displaystyle \sum _{i=1}^n\left(c_i{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_i}+c_i{\mathbf{t}}_{\mathbf{Y}}\right)}.$$ (3)

Now assume scalars c_i can be chosen so that

$${\displaystyle \sum _{i=1}^n{c}_i}{\mathbf{R}}_{{\mathbf{A}}_i}={\mathbf{0}}_{2\times 2}\text{where}{\displaystyle \sum _{i=1}^n{c}_i}=0.$$

Such equations may be represented as

$$\begin{gathered} {\mathbf{R}}_{\mathbf{A}}\mathbf{c}= \\ \left(\begin{array}{cccc}\cos \left({\theta }_1\right)& \cos \left({\theta }_2\right)& \cdots & \cos \left({\theta }_n\right)\\ \sin \left({\theta }_1\right)& \sin \left({\theta }_2\right)& \cdots & \sin \left({\theta }_n\right)\\ 1& 1& \cdots & 1\end{array}\right)\left(\begin{array}{c}{c}_1\\ c_2\\ ⋮\\ c_n\end{array}\right)={\mathbf{0}}_{3\times 1} \end{gathered}$$

where θ_i is the angle of rotation of ${\mathbf{R}}_{{\mathbf{A}}_i}$ representing the heading angle of the AGV fori = 1,2, … n. Note that a non-trivial solution to this system of equations is guaranteed to exist as long as n > 3, since the dimension of the associated nullspace is at least n−3. An orthonormal basis W for this nullspace can be calculated using the singular value decomposition (SVD) [40]. Note that each column of W formulates a c^T = (c c_{1 2}, …,c_n)^T such that R _Ac = 0. Therefore, an appropriate choice for the scalars may be represented as a linear combination of the columns of W.

For an appropriate W, the unknown R_Y may be calculated by reformulating Eq. (3) and noting that in the ideal case

$${\displaystyle \sum _{i=1}^n\left(c_i{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_i}-c_i{\mathbf{t}}_{{\mathbf{A}}_i}\right)}={\mathbf{0}}_{2\times 1}.$$ (4)

In the non-ideal case when Eq. (4) is not exactly equal to 0_{2 1×}, R_Y can be found by solving

$$\underset{{\mathbf{R}}_{\mathbf{Y}}}{\min }{\Vert {\mathbf{R}}_{\mathbf{Y}}{\mathbf{T}}_{\mathbf{B}}\mathbf{W}-{\mathbf{T}}_{\mathbf{A}}\mathbf{W}\Vert }^2$$

where

$$\begin{array}{l}{\mathbf{T}}_{\mathbf{A}}=\left({\mathbf{t}}_{{\mathbf{A}}_1}{\mathbf{t}}_{{\mathbf{A}}_2}\cdots {\mathbf{t}}_{{\mathbf{A}}_n}\right)\\ {\mathbf{T}}_{\mathbf{B}}=\left({\mathbf{t}}_{{\mathbf{B}}_1}{\mathbf{t}}_{{\mathbf{B}}_2}\cdots {\mathbf{t}}_{{\mathbf{B}}_n}\right).\end{array}$$

However, this problem is an example of an orthogonal Procrustes problem which reduces to solving

$$\underset{{\mathbf{R}}_{\mathbf{Y}}}{\max }\mathrm{tr}\left({\mathbf{R}}_{\mathbf{Y}}{\mathbf{T}}_{\mathbf{B}}\mathbf{W}{\mathbf{W}}^T{\mathbf{T}}_{\mathbf{A}}^T\right).$$

Note that this representation suggests that W does not need to be calculated via the SVD. Instead, just

$$\mathbf{W}{\mathbf{W}}^T={\mathbf{I}}_{n\times n}-\mathbf{O}{\mathbf{O}}^T$$

has to be computed where O is an orthonormal basis for the space spanned by the columns of ${\mathbf{R}}_{\mathbf{A}}^T$ [37]. This basis may be efficiently calculated via the Gram-Schmidt or QR decomposition for large n [40]. Then R_Y may be computed via the SVD as

$${\mathbf{R}}_{\mathbf{Y}}=\mathbf{V}\mathrm{diag}\left(1,\det \left(\mathbf{V}{\mathbf{U}}^T\right)\right){\mathbf{U}}^T$$

where

$$\mathbf{US}{\mathbf{V}}^T={\mathbf{T}}_{\mathbf{B}}\left({\mathbf{I}}_{n\times n}-{\mathbf{OO}}^T\right){\mathbf{T}}_{\mathbf{A}}^T.$$

Once R_Y is calculated then the other unknowns may be formulated by solving the linear system

$$\left(\begin{array}{ccc}{\mathbf{R}}_{{\mathbf{A}}_1}& & -\mathbf{I}\\ {\mathbf{R}}_{{\mathbf{A}}_2}& & -\mathbf{I}\\ & ⋮& \\ {\mathbf{R}}_{{\mathbf{A}}_n}& & -\mathbf{I}\end{array}\right)\left(\begin{array}{l}{\mathbf{t}}_{\mathbf{X}}\hfill \\ {\mathbf{t}}_{\mathbf{Y}}\hfill \end{array}\right)=\left(\begin{array}{c}{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_1}-{\mathbf{t}}_{{\mathbf{A}}_1}\\ {\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_2}-{\mathbf{t}}_{{\mathbf{A}}_2}\\ ⋮\\ {\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_n}-{\mathbf{t}}_{{\mathbf{A}}_n}\end{array}\right).$$ (5)

In conclusion, a closed-form solution may be formulated as

1. Formulate an orthonormal basis O for ${\mathbf{R}}_{\mathbf{A}}^T$.

2. Compute $\mathbf{M}={\mathbf{T}}_{\mathbf{B}}\left({\mathbf{I}}_{n\times n}-\mathbf{O}{\mathbf{O}}^T\right){\mathbf{T}}_{\mathbf{A}}^T$.

3. Set R_Y = V diag(1,det(VU^T)) U^T via the SVD of M.

4. Calculate t_{X Y}, t by solving the linear system of Eq. (5).

2.2. Multiple targets

Computing multiple target locations ${\mathbf{t}}_{\mathbf{X}}^j$, for j = 1,2,…,k, with respect to the world coordinate frame may aid in the accuracy of the calibration between the AGV and the robotic manipulator (see Fig. 1 where the four targets would be represented by ${\mathbf{t}}_{\mathbf{X}}^1,{\mathbf{t}}_{\mathbf{X}}^2,{\mathbf{t}}_{\mathbf{X}}^3,{\mathbf{t}}_{\mathbf{X}}^4$, respectively.). In this case, a generalization of the algorithm from the previous section can be made. Specifically, Eq. (3) can be generalized to

$${\displaystyle \sum _{i=1}^{n_j}\left(c_i^j{\mathbf{R}}_{{\mathbf{A}}_i}^j{\mathbf{t}}_{\mathbf{X}}^j+c_i^j{\mathbf{t}}_{{\mathbf{A}}_i}^j\right)}={\displaystyle \sum _{i=1}^{n_j}\left(c_i^j{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_i}^j+c_i^j{\mathbf{t}}_{\mathbf{Y}}\right)}.$$

for each i = 1,2, … n_j and j = 1,2,…,k. Then we set

$${\mathbf{R}}_{\mathbf{A}}=\mathrm{diag}\left({\mathbf{R}}_{\mathbf{A}}^1,{\mathbf{R}}_{\mathbf{A}}^2,\dots ,{\mathbf{R}}_{\mathbf{A}}^k\right)$$ (6)

$${\mathbf{R}}_{\mathbf{A}}^j=\left(\begin{array}{cccc}\cos \left({\theta }_1^j\right)& \cos \left({\theta }_2^j\right)& \dots & \cos \left({\theta }_{n_j}^j\right)\\ \sin \left({\theta }_1^j\right)& \sin \left({\theta }_2^j\right)& \cdots & \sin \left({\theta }_{n_j}^j\right)\\ 1& 1& \cdots & 1\end{array}\right)$$

$${\mathbf{T}}_{\mathbf{A}}=\left({\mathbf{t}}_{\mathbf{A}}^1{\mathbf{t}}_{\mathbf{A}}^2\cdots {\mathbf{t}}_{\mathbf{A}}^k\right)\mathrm{with}{\mathbf{t}}_{\mathbf{A}}^j=\left({\mathbf{t}}_{{\mathbf{A}}_1}^j{\mathbf{t}}_{{\mathbf{A}}_2}^j\cdots {\mathbf{t}}_{{\mathbf{A}}_{n_j}}^j\right)$$

$${\mathbf{T}}_{\mathbf{B}}=\left({\mathbf{t}}_{\mathbf{B}}^1{\mathbf{t}}_{\mathbf{B}}^2\cdots {\mathbf{t}}_{\mathbf{B}}^k\right)\text{with}{\mathbf{t}}_{\mathbf{B}}^j=\left({\mathbf{t}}_{{\mathbf{B}}_1}^j{\mathbf{t}}_{{\mathbf{B}}_2}^j\cdots {\mathbf{t}}_{{\mathbf{B}}_{n_j}}^j\right).$$

Here ${\theta }_i^j$ is the angle of rotation of ${\mathbf{R}}_{{\mathbf{A}}_i}^j$ for i = 1,2, … n_j and j = 1,2,…,k. Following the procedure from the previous section, the multiple target locations ${\mathbf{t}}_{\mathbf{X}}^j$ as well as the transformation Y can be found by

1. Formulate an orthonormal basis O for ${\mathbf{R}}_{\mathbf{A}}^T$.

2. Compute $\mathbf{M}={\mathbf{T}}_{\mathbf{B}}\left(\mathbf{I}-\mathbf{O}{\mathbf{O}}^T\right){\mathbf{T}}_{\mathbf{A}}^T$.

3. Set R_Y = V diag(1,det(VU^T)) U^T via the SVD of M.

4. Calculate ${\mathbf{t}}_{\mathbf{X}}^j,{\mathbf{t}}_{\mathbf{Y}}$ for j = 1,2,…,k by solving

$$\left(\begin{array}{cccccc}{\mathbf{R}}_{{\mathbf{A}}_1}^1& 0& \cdots & 0& 0& -\mathbf{I}\\ ⋮& & & & & \\ {\mathbf{R}}_{{\mathbf{A}}_{n_1}}^1& 0& \cdots & 0& 0& -\mathbf{I}\\ ⋮& & \ddots & & & ⋮\\ 0& 0& \cdots & 0& {\mathbf{R}}_{{\mathbf{A}}_1}^k& -\mathbf{I}\\ 0& 0& \cdots & 0& ⋮& -\mathbf{I}\\ 0& 0& \cdots & 0& {\mathbf{R}}_{{\mathbf{A}}_{n_k}}^k& -\mathbf{I}\end{array}\right)\left(\begin{array}{c}{\mathbf{t}}_{\mathbf{X}}^1\\ ⋮\\ {\mathbf{t}}_{\mathbf{X}}^k\\ {\mathbf{t}}_{\mathbf{Y}}\end{array}\right)=\left(\begin{array}{c}{\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_1}^1-{\mathbf{t}}_{{\mathbf{A}}_1}^1\\ ⋮\\ {\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_{n_1}}^1-{\mathbf{t}}_{{\mathbf{A}}_{n_1}}^1\\ ⋮\\ {\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_1}^k-{\mathbf{t}}_{{\mathbf{A}}_1}^k\\ ⋮\\ {\mathbf{R}}_{\mathbf{Y}}{\mathbf{t}}_{{\mathbf{B}}_{n_k}}^k-{\mathbf{t}}_{{\mathbf{A}}_{n_k}}^k\end{array}\right).$$

3. Results and discussion

3.1. Simulated data

Experiments were conducted using the single target method of Section 2.1 and the multiple target method of Section 2.2. The computed solutions were compared with solutions obtained from the Wu and Ren method shown in [4] and the iterative solution shown in [39]. It should be noted that the Wu and Ren method was only defined for the single target method in [4]. The method was generalized for the multiple target method using a technique similar to the one defined in Section 2.2. As the data was simulated, the transformation from the AGV to the robotic manipulator is known and was formulated as an offset of 10 mm in the x-position, 20 mm in the y-position, and an angular offset of 30 degrees. The AGV was simulated to reach eight positions as shown in Fig. 2. For the single target experiment, the robotic manipulator is supposed to reach a target located at (3000 mm, 3000 mm) in world coordinates. For the multiple target experiment, the robotic manipulator is supposed to reach four targets located at (3000 mm, 3000 mm), (3300 mm, 3000 mm), (3300 mm, 3300 mm), and (3000 mm, 3300 mm) in world coordinates.

Fig. 2.

Simulated setup. A mobile manipulator is programmed to reach multiple positions (red stars). The actual position of the mobile manipulator is denoted according to noise (black dots: high noise, blue square: medium noise, green circle: low noise). From each position, a mobile manipulator is to reach specific targets (red x). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Noise was added to the data to simulate high noise (α = 1), medium noise (α = 0.5) and low noise (α = 0.1), where the noisy heading angle was defined by adding αη degrees, the noisy 2D position of the AGV was defined by adding 50αη mm to each x− −, y component of the AGV, and the noisy 2D position of the target in robotic manipulator coordinates was defined by adding 0.1αη mm to each x− −, y component of the target position. Here η is defined as a pseudorandom value drawn from the standard normal distribution.

Results from the single target experiment are as shown in Table 1. The errors are defined as follows:

Table 1.

Comparison of the single target closed form solution (C) from Section 2.1, the Wu and Ren solution (W) shown in [4], and the iterative solution (I) shown in [39] on simulated data generated using one target. The errors are defined in terms of target (T) error, angle (A) error, and position (P) error.

	Low noise			Medium noise			High noise
	T	A	P	T	A	P	T	A	P
C	1.56	0.05	0.18	9.25	0.27	1.49	18.90	1.56	9.57
W	1.72	0.10	0.17	9.74	0.04	1.61	18.19	2.36	10.00
I	1.13	0.03	0.24	7.43	0.30	3.69	24.60	1.57	9.76

target error (T) = ‖computed target−(3000,3000) ‖

angle error (A) = |computed angle−30 |

position error (P) = ‖computed position−(10,20) ‖.

Results between the single target closed form solution (C) and the Wu and Ren method (W) are comparable. However, the results may vary from the iterative solution (I).

Results from the multiple target experiment are as shown in Table 2. The errors are defined as for the single target except now the computed target represents four target positions – (3000 mm, 3000 mm), (3300 mm, 3000 mm), (3300 mm, 3300 mm), and (3000 mm, 3300 mm) – and thus the target error (T) is updated accordingly. As with the single target experiment, the multiple target closed form solution (C) and the Wu and Ren method (W) are comparable, while the iterative solution (I) may vary.

Table 2.

Comparison of the multiple target closed form solution (C) from Section 2.2, the Wu and Ren solution (W) shown in [4], and the iterative solution (I) shown in [39] on simulated data generated using four targets. The errors are defined in terms of target (T) error, angle (A) error, and position (P) error.

	Low Noise			Medium Noise			High Noise
	T	A	P	T	A	P	T	A	P
C	1.20	0.07	0.33	11.81	0.14	2.88	17.42	0.71	6.02
W	1.20	0.05	0.31	11.85	0.18	2.85	20.22	0.77	6.12
I	2.19	0.05	0.47	12.32	0.40	3.78	8.34	0.71	3.05

3.2. Real data

The algorithms from Section 2 were applied to data gathered in a laboratory setting at the National Institute of Standards and Technology (NIST) (see Table 5 in Appendix A). As test cases, two types of mobile manipulator performance measurements have been studied in [2,3]: static and index manipulator base configurations. The static case allows the AGV to position the manipulator base to a single location where it can reach all chosen assembly targets within a pattern (circle, square, or other). The index case allows the AGV to reposition the manipulator base as needed near the RMMA to reach all assembly targets in multiple patterns on the RMMA. As manipulators have become more collaborative, their lengths are approximately 1 m ± 0.5 m. The size of the RMMA was chosen to be 1.2 m × 0.6 m to allow the approximately 1 m manipulator used for these tests to reach an entire approximately 457.20 mm square and a 304.80 mm diameter circle. The distance between the square and circle patterns of 152.40 mm forces the mobile manipulator to index between patterns as a different performance measurement test case from the static case. The RMMA has predefined holes with a tolerance of 0.25 mm.

For this experiment, an AGV was moved to 10 stops from which a robotic manipulator tried to reach the center of the four targets (see Fig. 3). The 2D position of each of the targets reached by the robotic manipulator was recorded as well as the 2D position and heading angle of the AGV (see Table 5 in Appendix A). From this data, the location of each of the targets in the world coordinate system was calculated by the single target closed form solution presented in Section 2.1 as well as the multiple target closed form solution presented in Section 2.2. These calculations were then compared with the Wu and Ren closed form solution [4] and the iterative solution presented in [39].

Fig. 3.

Real target setup. The four targets (reflectors) lie on a straight line such that the ground truth distance between two successive targets is 457.20 mm 152.40 mm, and 304.80 mm, respectively.

The single target closed form solution was used four times to determine the target location in world coordinates. This was formulated by first gathering the data from Table 5 in Appendix A. For example, target 1 was reached by the robotic manipulator at Stops 2, 4, 6, 8, 9, and 10. Then the corresponding AGV data was used to perform the calibration which resulted in the target 1 location in the world coordinate system. A similar procedure was used to determine target 2, target 3, and target 4 positions in the world coordinate system. To determine the accuracy of the single target closed form solution, the distance between the successive targets was compared with the Wu and Ren closed form solution [4] and the iterative solution presented in [39]. The results are shown in Table 3.

Table 3.

Comparison of the single target closed form solution from Section 2.1, the Wu and Ren solution [4], and iterative solutions [39] on data from Table 5 in Appendix A. The ground truth distance between two successive targets is 457.20 mm, 152.40 mm, and 304.80 mm, respectively (see Fig. 3).

Ground truth Dist (mm)	Closed Form		Wu and Ren		Iterative
	Dist	Err	Dist	Err	Dist	Err
457.20	459.84	2.64	453.35	3.85	461.40	4.20
152.40	152.66	0.26	152.22	0.18	151.50	0.90
304.80	305.23	0.43	300.94	3.86	305.32	0.52

The multiple targets closed form solution was used just one time to determine all four target locations in world coordinates. This was formulated by first gathering the data from Table 5 in Appendix A to build the matrices of Eq. (6). For example, ${\mathbf{R}}_{\mathbf{A}}^1$ is of size 3 × 6 since the robotic manipulator reached target 1 at only 6 stops. To determine the accuracy of the multiple target closed form solution, the distance between the successive targets was compared with the Wu and Ren solution shown in [4] as well as iterative solution presented in [39]. The results are shown in Table 4. Notice that the differences between the three solutions are less than the single target solution shown in Table 3 suggesting that multiple targets aid in the accuracy of the calibration.

Table 4.

Comparison of the multiple targets closed form solution from Section 2.2, the Wu and Ren solution [4], and iterative solutions [39] on data from Table 5 in Appendix A. The ground truth distance between two successive targets is 457.20 mm, 152.40 mm, and 304.80 mm, respectively (see Fig. 3).

Ground truth Dist (mm)	Closed Form		Wu and Ren		Iterative
	Dist	Err	Dist	Err	Dist	Err
457.20	456.07	1.13	456.07	1.13	457.90	0.70
152.40	152.67	0.27	152.70	0.30	151.50	0.90
304.80	303.94	0.86	303.94	0.86	302.51	2.29

4. Conclusion

This paper outlines a closed form solution for solving the positional component of the horizontal two-dimensional robot-world/hand-eye calibration problem in order to determine a target’s location with respect to a fixed world coordinate system. This solution is then generalized to the case where multiple target locations are needed. These solutions were applied to simulated data as well as to data gathered in a laboratory setting at NIST to formulate the location of fixed targets with respect to a common world coordinate frame. These target locations were compared with locations computed from an iterative solution as well as an existing closed form solution. The results from the closed form solution formulated in this paper were found to be highly reliable and accurate.

Appendix A

Table 5.

Experimental data from NIST Lab where an AGV is moved to 10 stops and searches for four target positions.

	Robotic manipulator			AGV
Stop	x (mm)	y (mm)	target	x (mm)	y (mm)	θ (degrees)
1	−12.66	−920.39	2	7151	11,809	90.14
	−10.90	−768.67	3
	−6.14	−463.91	4
2	136.72	−559.10	1	6092	14,917	314.65
	−190.23	−879.53	2
	−298.18	−987.30	3
3	−249.41	−623.45	4	5701	13,371	359.81
	56.70	−627.04	3
	208.79	−628.66	2
4	166.98	−630.11	1	5701	13,866	359.74
	−292.38	−626.01	2
	−443.82	−624.90	3
5	−158.24	−640.77	4	5999	12,191	44.65
	56.32	−857.71	3
	165.35	−966.33	2
6	903.20	−204.80	1	8070	12,998	90.38
	912.17	252.93	2
	916.07	406.40	3
7	54.31	−649.58	4	8259	12,113	136.15
	−168.54	−856.82	3
	−279.16	−961.42	2
8	−144.45	−553.30	1	8259	14,883	224.35
	179.82	−877.20	2
	288.62	−984.83	3
9	−872.43	−141.94	4	8040	14,092	270.04
	−868.01	163.89	3
	−865.18	317.42	1
10	−11.20	−364.14	1	7166	15,213	270.14
	−16.73	−821.00	2
	−20.05	−973.87	3
	−25.21	−1276.75	4