Validation of the network method for evaluating uncertainty and improvement of geometry error parameters of a laser tracker

Abstract

Five National Metrology Institutes (NMIs) worked together to validate the assessment of laser tracker’s (LT) uncertainty for large scale dimensional metrology in subsequent measurements using the network method. The LT uncertainty is assessed by measuring a set of a fixed network of targets from different LT positions. Afterward, we must perform a ”bundled adjustment” of all the measurements to determine the transformations of the LT positions that minimize the residuals (differences) between a computed ”virtual” group of targets, called composite points, and the redundant targets coming from the LT positions transformed to a unique LT position. Each residual is weighted with the LT’s Maximum Permissible Error (MPE) specified by the LT manufacturer. The standard deviation of the weighted residuals becomes LT uncertainty. Afterward, with Monte-Carlo simulation, we propagate the LT uncertainty to the position of every target in the network and 3D distances between them. LT uncertainty is valid if 3D distance’s uncertainties computed with composite points is greater than the error computed with these distances minus the same distances calibrated with the line of sight method (LOS) or if the absolute normalized error is less or equal to one. LOS method uses only the LT’s interferometer for calibration, which dispense the use of a calibrated scale bar.

Based on the network method, the NMIs developed two methods to improve LT geometry error parameters. The first improves the LT geometry error parameters by fitting the composite points to the LT geometry error model. The second evaluates the parameters placing the LT geometry error model in the minimization of the bundled adjustment’s residual. LT geometry error parameters are valid if the parameter’s uncertainties are negligent.

1. Introduction

One of the essential tasks of a National Metrology Institute (NMI) is to report uncertainty in measurements according with GUM (Guide to the Expression of Uncertainty in Measurement), see [1]. This task is not easy to perform, and this is why although NMIs report uncertainty, much of the industry that makes any kind of measurements around the world do not report uncertainty. What the industry might not know is that not reporting uncertainty can represent monetary losses, low quality of its products, loss of product safety, etc. The present work explores and validates the network method to report uncertainty in dimensional measurements using a laser tracker (LT) and improves the geometry error parameters of the LT.

In response to the project’s call of the Sistema Inter-Americano de Metrología (SIM) project with the Inter-America Development Bank (IADB) on strengthening NMIs in the Hemisphere, five NMIs joined and submitted the research proposal entitled ”large-scale dimensional metrology”. One of the main objectives of the proposal was the measurement uncertainty assessment in large scale dimensional metrology. The NMIs focused on this objective and decided to select an LT to perform the measurements. The selection of the LT was because it is a widely used instrument in large-scale dimensional metrology, and all NMIs have such instrument. However, any other instrument for large scale dimensional metrology can use the network method to report uncertainty. The objective of the work was then focused on LT measurement uncertainty and its validation, and the evaluation and improvement of actual LT’s geometric errors parameters. This is a significant challenge because we must evaluate the task’s uncertainty and compare it against the dimensional tolerance of the piece to be measured subsequently. The reason to also include the evaluation and improvement of the LT geometric errors is just for knowing the status of the LT (its actual performance evaluation), something that has not been explored in other works to best of our knowledge. We should note that it does not matter if the LT is well or poorly compensated because the evaluated LT uncertainty should include any random or systematic error.

To comply with the challenge, the five NMIs decided to use the LT uncertainty evaluation with the network method introduced by Calkins’s dissertation, [2] and Calkins et al. [3]. The network method consists of measurement of a set of fixed targets (three-dimensional (3D) points) from different LT positions. Because of this, the NMIs measured a series of fixed targets located spatially within a common volume of measurement for all NMIs and with similar environmental conditions. The network of targets included 19 fixed targets within a defined volume, which were measured by all NMIs in their laboratories. Fifteen targets were distributed within ±200 mm of their desired 3D location and four of them were inside the same volume but aligned to get three calibrated distances using a line of sight method (LOS). LOS method uses mainly the LT’s interferometer which has less uncertainty and avoids the use of LT encoders that are less accurate, Wang et al. [4]. The LOS method gives us the chance to dispense a calibrated scale bar, so the NMIs don’t need to buy a calibrated artifact, instead, the same LT can be used for distances calibration.

LT measurement uncertainty for each NMI was evaluated using a commercial software, [5] and a custom self-developed software for validation of the first. Both software used the data from measurements on the network of targets (or 3D points) from five different LT positions (it means five different coordinate systems defined on the base of each LT position). Both software uses the bundled adjustment method, [6]. This method can transform the 19 measured targets from five different LT positions to a unique LT position; in our case, it is the first LT position. From this unique position, a new composite group of targets (nonlinear least square points) is computed by minimizing the residuals (differences) between these composite targets and those transformed to the unique LT position (the minimization involves the movement of the composite points and LT positions already transformed to a unique position). Each residual is weighted with the LT’s Maximum Permissible Error (MPE) specified by the manufacturer reflecting in this way the quality of the data points delivered by the LT. The standard deviations of the weighted residuals are considered as the uncertainty of every LT variable: range (r), horizontal angle (θ), and vertical angle (ϕ). Afterward, with Monte-Carlo simulation [7], we can propagate the uncertainty of each LT variable to the position of every target in the network or to 3D distances between them.

Calkins [2], compared some LT under different environments using the uncertainty of each LT variable evaluated with residuals between composite points and the transformed measured points from each LT position to a unique LT position. Unlike Calkins, in this paper, the LT uncertainty is compared in similar environmental conditions with a similar setup of network of targets.

What is novel in this work is that the LT uncertainty evaluated with the network method is validated for subsequent large scale dimensional measurements. As in the case of Calkins [2], NMIs also consider the fixed points as reference to evaluate LT uncertainty. However, this is not enough, so we decided to include at least three calibrated distances with the LOS method that would serve as the validation for the evaluated LT uncertainty. If the estimated distance’s uncertainty computed with composite points (corrected and uncorrected, see red circles in fig. 4) and LT uncertainty propagation, is greater than the error evaluated between the composite points and the calibrated distances between the same points, then the LT uncertainty is validated, see documentary standard ISO 15530-4:2008 [8] . It means that the LT uncertainty covers any source of random and systematic errors coming from different factors. Another parameters used to validate LT uncertainty were the absolute normalized error see documentary standard ISO 17043 [9], and standard deviation of the points. We will explain these parameters later. The use of the LOS method to avoid using a calibrated scale bar is also novel in this validation.

Figure 4:

Composite point i, ${\overrightarrow{C}}_i$ red circle), rotation and translation between LT positions since LT position 1 ${\left[R,\overrightarrow{t}\right]}_{2,3}^1$ and the residuals for each LT variable ϵ_ri,j, ϵ_θi,j, ϵ_ϕi,j. ${\overrightarrow{P}}_{1,j}$ are already rotated and translated to LT position 1.

Another contribution of this work is the implementation of two methods based on the network for LT error geometry parameters improvement. In the first method, that we call the least square method (LS), we use the composite points coming from the network of targets to evaluate the geometrical error parameter of every LT and improve those parameters. With the improved parameters, we can compensate for the LT point measurements and thus reduce the point coordinate uncertainty. Wang et al. [10] proposed a method to correct terrestrial laser scanners using the LS method. Our proposal use the same least-square approach, but composite points, front-face, and back-face LT measurements are the reference to identify systematic errors of LTs. In the second method, that we call nonlinear least square method (NLLS), the LT error geometry parameters are part of the residual minimization of the bundled adjustment, it means, the minimization involves iterations that moves not only the composite points, and LT position transformation, but the LT parameters.

With both methods, we can identify a poor LT performance, afterward, in a real measurement the user may receive a warning recommending that the LT must be compensated; in our case, we compensate the 3D points offline. Both methods uses the geometrical error models described by Loser and Kyle, [11], and Muralikrishnan et al., [12].

For LT geometrical error parameters validation, we use the front-face and back-face measurements, if the differences, tendencies or offsets between both decreases and are around zero after compensation then the evaluated parameters are valid. For the evaluation of parameters with the nonlinear method, we also evaluate the uncertainties of the parameters, and if the parameter’s uncertainties are negligent, then the LT parameters are valid. Documentary standards like [13], [14], and [15] suggest front-face and back-face measurement to identify a poor LT performance, but they don’t explain how to compensate for it. The proposed methods described here shows how to do it.

Conte et al. [16] describe different techniques to find LT geometry error parameters using calibrated distances and reference points obtained from a CMM. They conclude that distance minimization is the best option to determine LT systematic errors. In our case, because we have composite points and residuals, we used these points and residuals as reference for the minimization and evaluation of geometrical error parameters.

Hughes et al. [17], uses the network method to find LT geometrical error parameters. Still, their approach is different from our first method (LS) to evaluate LT geometry error parameter because they estimate the parameters without having reference points as part of the evaluation. Our second method (NLLS) differs from Hughes’s work in the evaluation of the parameters and uncertainties of those parameters. We use partially the set up proposed by Hughes et al. [17] for the target distribution inside the network.

1.1. Laser tracker for large scale dimensional metrology

An LT is a measuring instrument that tracks the movement of a retroreflector called spherically mounted retroreflector (SMR) and calculates its position in spherical coordinates. The distance to the SMR (R_m) can be measured by an interferometer (IFM) or by an absolute distance meter (ADM). In contrast, the zenith angle V_m and the azimuth angle H_m are measured by two angle encoders. The SMR returns the laser beam, and it falls on a position sensor detector (PSD) that detects changes in position and activates the axes of movement of the LT so that the beam always strikes the center of the SMR, Fig. 1.

Figure 1:

Measuring principle of an LT.

The different manufacturers of LT deliver a software package with procedures to compensate for its systematic errors so that the instrument is working within its maximum permissible error (MPE). To ensure that the instrument in fact meets its MPE, performance evaluation of this equipment is done by applying test procedures described in documentary standards. Examples of documentary standards for performance evaluation of LT are ISO 10360-10 [13], ASME B89.4.19 [14], and VDI/VDE 2617-Part 10 [15]. The use of these documentary standards allows us to know if the instrument meets or does not meet the MPE. Still, no quantification of systematic errors or uncertainty dedicated to the task is obtained. Under these circumstances, the question of whether it is possible that the MPE or the results from performance evaluation can become the measurement uncertainty of the piece to be measured subsequently arises.

The evaluation software used in this work by default uses the instrument’s MPE as weights for the computed residuals (differences between measured LT points and LT composite points) to evaluate the LT uncertainty if the network method is applied. Weighting those residuals means that the quality of the data points delivered by the LT is affected by its actual MPE, and then the uncertainty reported is also affected.

For more information on LT, we recommend reading Muralikrishnan et al. [18] who survey the literature in all areas of LT as applied to large scale dimensional metrology (LSDM), with emphasis on error modeling, measurement uncertainty, performance evaluation and standardization.

As part of LT uncertainty evaluation, we must fit the traceability of the instrument. In our case, using the LOS method, the traceability is conformed if the LT interferometer is calibrated against the national reference length.

The network method was validated if the evaluated uncertainty is greater than any LT systematic or random error. The geometrical errors are validated if the differences between front face and back face measurements decreases after LT correction.

In the rest of the article, we describe our approach for LT uncertainty estimation in Section 2. This Section includes an introduction to the method, the setup, and the procedure to evaluate LT uncertainty and LT geometrical error parameters. Afterward, we discuss NMI’s results and LT uncertainty comparison and validation in Section 3. Finally, we conclude summarizing our findings and delineating future directions of inquiry.

2. Laser Tracker’s uncertainty

2.1. Introduction

As explained before, the LT uncertainty evaluation is based on the network method introduced in Calkins dissertation, [2] and Calkins et al. [3]. The method evaluates the LT uncertainty at the site where the measurement is made, for this, the targets must be placed around or on the piece under measurement, and their 3D position, regardless of the LT position, must not change during the measurement. Any change in the position of the targets is converted into the factors that influence the uncertainty of the LT measurement, such as the user, environmental conditions, fixations, among others. The network method can be used to evaluate uncertainty in other measurements with the appropriate adjustments.

2.2. Setup for the network of points and laser tracker

Each NMI used its LT to perform the measurements of a 3D point network. The nominal 3D points of the network with respect to the initial LT position (LT1) where the same for all participants, see Fig. 2. These nominal points are shown in Table 1. The setup is based partially in the configuration shown in Hughes et al. [17].

Figure 2:

Targets field and LT positions setup, all in meters.

Table 1:

Nominal points from initial LT position, LT1.

Point Number	X [m]	Y [m]	Z [m]
1	5.00	0.00	−0.90
2	5.00	0.00	0.10
3	5.00	0.00	1.10
4	3.50	1.00	−0.90
5	2.00	0.00	−1.12
6	2.00	−1.50	−0.70
7	1.00	1.00	−0.90
8	0.50	0.00	−0.30
9	0.00	−1.50	−0.50
10	−0.50	−1.50	−0.90
11	−0.50	−1.50	0.10
12	−0.50	−1.50	1.10
13	0.00	1.00	−1.10
14	5.00	1.00	0.60
15	−0.50	1.20	0.60
16	4.77	1.20	−0.13
17	4.77	0.40	−0.13
18	4.77	−0.40	−0.13
19	4.77	−1.20	−0.13

The 19 fixed targets, were measured from five LT locations, Fig. 2 shows the measurement setup. An arbitrary LT position was named as LT1 from which all 19 targets were measured. This is the origin point of the coordinate system (point 0 in Fig. 2), and the coordinate system is coincident with the drawn axes. LT position 2 (LT2) is the same as position 1 but rotated 180° around the Z – axis. Position 3, 4, and 5 can be seen in red triangles in Fig. 2 (LT3, LT4, and LT5). With respect to LT1 the nominal LT position were:LT3 : X = 1000 mm, Y = 0 mm, Z = −400 mm; LT4 : X = 2000 mm, Y = 1000 mm, Z = −200 mm; LT5 : X = 4000 mm, Y = −1000 mm, Z = 200 mm. Due to the difficulty in establishing these nominal positions for the LTs, a tolerance of ±200 mm 3D position was accepted. Since the instrument will be moved throughout the workspace, most of the targets should be visible from all instrument locations.

For LT uncertainty validation, the points 16 to 19 were positioned in line and about 800 mm from each other. The linear positioning allows the possibility to measure them using the line of sight (LOS) method, Wang et al. describe this method, [4].

Two types of LTs were used: one with laser in the head and the other with laser in the column. Each LT had its own environmental compensation system. Instead of the use of the LOS method, one of the NMI used a laser tracer to calibrate the distances between points 16 to 19. The laser tracer works like a laser interferometer with two degrees of freedom. Two NMIs used a nominal 1 m calibrated bar in points 7 and 13 as part of the 3D network of targets. According to Hughes et al. [17], to evaluate the LT birdbath error appropriately, a diagonal line of sight must be included in the setup. Because of this, points 5 and 8 are aligned in this way, and LT in positions LT1 and LT3 are aligned with these points, with LT1 outside the line and LT3 inside the line.

The environmental conditions at each NMI location were around 20°C±1 °C. The targets were fixed using tripods, stable stands, on walls, floor, etc. An essential condition for the targets is that these must be as stable as possible. Glues based on cyanoacrylate are recommended to attach the target nests as they have minimal drift. Some NMIs had only one SMR, which must be placed sequentially on the 19 targets to perform the measurements. Other NMIs had more than one SMR, so they were placed on some target nests, thus reducing the movements.

2.3. Procedure to evaluate and validate LT uncertainty and LT geometrical errors parameters using the network method

The procedure to assess and validate LT uncertainty and LT geometrical error parameter is shown in the flow diagram of Fig. 3.

Figure 3:

Workflow to evaluate and validate LT uncertainty and LT geometrical error parameter using the network method. LT means Laser Tracker, NP network of points, F/B means Front minus Back measurements, R/T means Rotation and Translation of LTs to LT pos 1. SA is the software Spatial Analyzer, U(CD) is uncertainty of the composite distances and U is LT’s uncertainty. SOFT, means Software.

2.3.1. Step 1

Once the setup is finished, each laboratory starts an LT field check to ensure that the LT is compensated and working within manufacturing specifications. The field check is carried out with the maintenance software of each LT. If the field check fails, LT compensation must be performed with the same maintenance software until the LT passes the field check.

2.3.2. Step 2

After this, to ensure that the targets are fixed stably, and to know the magnitude of the target’s drift, repeatability tests were carried out. All targets were measured sequentially from LT position one at least ten times in front-face mode only. If some targets were not fixed properly, we address that issue before starting the next step in the workflow. If some targets had to be adjusted, then the repeatability test must be done again before proceeding.

2.3.3. Step 3

After repeatability measurements, the measurements of all 19 targets began in the front-face and back-face mode for each point. Once measurements from a position are completed, the LT is moved to the next position, and the measurement process is repeated until all points are measured from the five LT positions. This results in 190 measured points, 95 in the front-face, and 95 in the back-face.

With all the measured points from each LT position, the bundled adjustment method can be performed using the averaged coordinate points between the front-face and back-face measurements in each LT position. The reason to use the averaged front-face (f) and back-face (b) measured points is because this average still retain the LT systematic errors for the first method (called least square method) that evaluates the LT geometry error parameters. If we use the 190 points and 10 LT positions, then the LT systematic errors are removed by the bundled adjustment when it evaluates the reference composite points. For the second method (called no linear least square method) that evaluates the LT geometry error parameters, we could use five LT positions and 95 averaged measurements, or 10 LT positions and 190 measured points indistinctly. The objective function of the bundled adjustment optimization process for the network method and as a consequence the calculation of composite points is shown as follows:

$$\mathrm{arg\; min}f\left(T_{\overrightarrow{t},R},{\overrightarrow{C}}_i\right)=\mathrm{arg\; min}\sum _{j=1}^{2\times m}\sum _{i=1}^n{\left({\overrightarrow{w}}_{i,j}·{\overrightarrow{ϵ}}_{i,j}\right)}^2$$ (1)

where:

$T_{\overrightarrow{t},R}$ is the set of with three translation and three rotation parameters of target coordinate transformation from each LT position to the reference position (the reference position can be usually one of the j LT position, where j is the LT position). There are 6 rotation and translation parameters per LT position for a total of (m − 1) × 6, where m is the number of LT positions. As mentioned earlier, the frame of the first LT position is the reference/world coordinate system;

${\overrightarrow{C}}_i$ is the n×3 matrix of composite coordinates of the i targets in the reference position, where n is the number of targets and i is the target number; ${\overrightarrow{ϵ}}_{i,j}$ are the residual errors along the ranging direction (r), the horizontal angle (θ) and vertical angle directions (ϕ) for the i — th target measured from the j — th LT position. 3 dimensions for a LT (eg.,r, θ, ϕ), ${\overrightarrow{ϵ}}_{i,j}$ is defined in equation 2.

$${\overrightarrow{ϵ}}_{i,j}={\overrightarrow{C}}_i-{\overrightarrow{P}}_{i,j}$$ (2)

where: ${\overrightarrow{C}}_i$ are the composite coordinates points; and ${\overrightarrow{P}}_{i,j}$ are the rotated-translated points i from each LT position j into LT position one;

${\overrightarrow{w}}_{i,j}$ are the weights assigned to the residuals along r, θ, and ϕ for the i — th target measured from the j — th LT position. The initial weights used for the optimization may be generated from the MPEs of the LT, in our case the MPEs values were 7.62μm + 2.5μm/m for r, and 1 arc seconds for θ and ϕ, with k — 1. To get the weights each ${\overrightarrow{P}}_{i,j}$ was multiplied by its respective MPE value.

To avoid local minimization of equation 1, a starting point for $T_{\overrightarrow{t},R}$ can be a best-fit between the points in LT position j and its respective points in LT position one. The initial conditions for ${\overrightarrow{C}}_i$ can be an average between the points in LT position j rotated and translated to LT position one.

Fig. 4 shows the concept of composite points ${\overrightarrow{C}}_i$ (red circle, that is essentially a non linear least square point evaluated with the rotated and translated points measured by each LT at different positions), the rotated-translated points i from each LT position j into LT position one, ${\overrightarrow{P}}_{i,j}$, rotation, and translation between LT positions j to LT position 1, ${\left[R,\overrightarrow{t}\right]}_j^1$, and the residuals i for each LT variable at each LT position j: ϵ_ri,j, ϵ_θi,j, and ϵ_ϕi,j.

Once the composite points ${\overrightarrow{C}}_i$ are computed with equation 1, we can use equation 2 to compute all residuals errors ${\overrightarrow{ϵ}}_{i,j}$. If we attribute these errors to uncertainty in the measurement device, we can statistically process these values to determine an uncertainty for the device. There are many statistical methods that could be used to accomplish this. For this analysis, the standard deviation of the residuals was chosen. In this case, we assume the mean to be zero, since this is the value at which the measurements perfectly represent the best-fit point. The uncertainty of a measurement component (LT uncertainty) is,

$${\overrightarrow{u}}_{LT}=\sigma \left({\overrightarrow{ϵ}}_{i,j}\right)or{\overrightarrow{u}}_{LT}=\sqrt{\frac{1}{i\times j-1}\sum _{j=1}^m\sum _{i=1}^n{\left({\overrightarrow{ϵ}}_{i,j}\right)}^2}$$ (3)

${\overrightarrow{u}}_{LT}$ is the overall LT uncertainty for a measurement component (eg.,r, θ, ϕ), i is the number of points, and j is the number of LT positions.

For LT uncertainty convergence, we could repeat equations 1 and 3, n times, in each iteration the evaluated LT uncertainty or standard deviation of the residuals could be used as new weights in equation 1 for the next iteration and repeated until the LT uncertainty do not change within a defined tolerance. In the results that we show only one iteration was done taking off the possible changes of ${\overrightarrow{u}}_{LT}$.

In the case of a spherical measurement device, equation 3 is applied to all 3 measurement values. This results in 3 scalar uncertainty values: ${\overrightarrow{u}}_{LT}$. These values represent the uncertainty values for the measurement components that encompass one sigma or approximately 68 % of the observed residual errors. Other statistical methods could also be used. The objective is to represent the remaining residual errors for each component as a single value, see these values in Table 4.

Table 4:

${\overrightarrow{u}}_{LT}$, LT uncertainty comparison for all NMIs, letter B and D are for LT of the same type. LS means least square method and NLLS means nonlinear least square. N.E. means not evaluated. All evaluated values are k = 1.

Lab	LT type	LT Location	${\overrightarrow{u}}_{LT}$ with and without LS compensation			${\overrightarrow{u}}_{LT}$ with NLLS compensation
			Horizontal (θ)	Vertical (ϕ)	Range (r)	Horizontal (θ)	Vertical (ϕ)	Range (r)
			(arc sec)	(arc sec)	(mm)	(arc sec)	(arc sec)	(mm)
NIST Typical and initial values from MPE data sheet for all LTs: (r) = 7.62μm+2.5μm/m; (θ)=(ϕ)= 1 arc sec (k = 1)	A	1	0.4	0.4	0.007	0.4	0.4	0.007
		2	0.3	0.4	0.005	0.4	0.4	0.003
		3	0.4	0.4	0.005	0.4	0.5	0.003
		4	0.6	0.4	0.004	0.6	0.4	0.005
		5	0.5	0.4	0.004	0.7	0.5	0.004
		Average	0.4	0.4	0.005	0.5	0.5	0.005
	B1	1	0.8	0.5	0.010	0.4	0.4	0.003
		2	0.8	0.5	0.010	0.4	0.3	0.003
		3	0.8	0.5	0.008	0.3	0.4	0.002
		4	0.7	0.6	0.010	0.0	0.5	0.004
		5	0.6	0.5	0.007	0.4	0.4	0.004
		Average	0.7	0.5	0.009	0.4	0.4	0.003
	C	1	0.4	0.2	0.004	0.4	0.2	0.004
		2	0.4	0.3	0.003	0.4	0.3	0.003
		3	0.4	0.4	0.004	0.4	0.4	0.004
		4	0.4	0.5	0.003	0.4	0.4	0.004
		5	0.4	0.4	0.004	0.4	0.3	0.004
		Average	0.4	0.4	0.004	0.4	0.3	0.004
INTI	D1	1	1.0	0.8	0.015	0.9	0.8	0.017
		2	0.6	0.5	0.010	0.5	0.5	0.012
		3	1.2	1.0	0.011	1.3	0.8	0.011
		4	1.2	0.7	0.019	1.3	0.5	0.014
		5	1.0	1.0	0.019	1.0	1.0	0.024
		Average	1.0	0.8	0.014	1.0	0.7	0.015
CENAM	B2	1	2.0	1.3	0.012	1.5	1.4	0.013
		2	1.6	1.1	0.020	1.3	1.3	0.017
		3	1.4	1.2	0.014	1.6	1.4	0.014
		4	1.6	1.5	0.016	1.7	1.6	0.020
		5	1.6	1.9	0.015	1.5	1.9	0.018
		Average	1.6	1.3	0.015	1.5	1.5	0.016
INMETRO	B3	1	1.0	1.0	0.020	2.0	1.8	0.037
		2	1.0	0.8	0.021	1.6	1.7	0.048
		3	1.1	0.9	0.021	2.4	0.7	0.026
		4	0.7	0.9	0.018	2.2	1.5	0.029
		5	1.1	0.8	0.016	3.3	2.3	0.045
		Average	1.0	0.8	0.019	2.3	1.6	0.037
INACAL	D2	1	2.9	2.9	0.049	N.E.	N.E.	N.E.
		2	3.8	2.1	0.022	N.E.	N.E.	N.E.
		3	2.3	1.8	0.055	N.E.	N.E.	N.E.
		4	2.5	2.8	0.042	N.E.	N.E.	N.E.
		5	2.9	2.4	0.044	N.E.	N.E.	N.E.
		Average	2.8	2.4	0.042	N.E.	N.E.	N.E.

In this step of the workflow, we evaluate whether we can use all network of targets or remove outliers; at this time, we can remove outliers in SA and perform the evaluation of composite points ${\overrightarrow{C}}_i$, R and $\overrightarrow{t}$ of LT positions with respect to position one. The outliers represent targets that, for some reason, moved during or before the measurement. At the moment, our custom self-developed script requires all the points from the network or to remove them prior to evaluation. Results sections will show assessments eliminating outliers.

Either before or after we evaluate the composite points ${\overrightarrow{C}}_i$, and R, $\overrightarrow{t}$ of each LT with respect to LT position one, we assess the differences between the front-face and back-face measurements (f-b) of each X Y Z coordinate target in the network. If a tendency or offset is founded for the X Y Z coordinate differences (i.e., f-b differences in all LT positions are not around zero) these tendencies or offsets must be eliminated before proceeding with the uncertainty evaluation and validation of each NMI’s LT (it means and improvement of geometry error compensation parameters of the LT). For this purpose, the LT error parameter evaluation, a compensation of network of targets with these error parameters, and the evaluation of new composite points must be performed again (we will call this new composite points as corrected composite points).

2.3.4. Step 4

This step in the workflow helps to identify poor performance of an LT (it means that step 1 was not done correctly) and improve it. However, this led us to the question of how to make the compensation of the LT and hence of the network of targets. The answer lies in the error models introduced by Loser et al. [11] for LT with laser in the column and Muralikrishnan et al. [12] for LT with laser in the head; these are shown in equations 5 to 10.

As already mentioned for LT error parameter evaluation we have two methods: the first one called least square method for geometry errors parameter evaluation, and the second, called nonlinear least square method.

Least square (LS) method for LT geometry error parameters evaluation.

In this method, we make the supposition that the composite points C_i evaluated with average front-face and back-face points from each LT position are the reference points (R_c, H_c, and V_c for equations 5 to 10). Next, with the measured front-face and back-face points from each LT position, we can formulate a system of equations of the form Ax = b and solve for the error parameters x for each LT. Matrix A includes the measured values in front-face and back-face (R_m, H_m, and V_m for equations 5 to 10), and vector b consist of the composite points ${\overrightarrow{C}}_i$ repeated for front-face and back-face. The error parameter x is solved by equation 4 in a least-square sense:

$$x={\left(A^{T}A\right)}^{-1}{A}^{T}b$$ (4)

The set of parameters to correct the measured values of an LT with a laser in the head is showed in the following equations:

$$R_c=R_m-x_{11}-x_2\mathit{sin}\left(V_m\right)$$ (5)

$$\begin{gathered} H_c=H_m-x_{12b}\mathit{cos}\left(2H_m\right)-x_{12a}\mathit{sin}\left(2H_m\right) \\ +k\left[\frac{x_{1t}}{R_m\mathit{sin}\left(V_m\right)}+\frac{x_{6t}}{\mathit{sin}\left(V_m\right)}+\frac{x_8}{\mathit{tan}\left(V_m\right)}+x_{9x}\mathit{cos}\left(H_m\right)-x_{9y}\mathit{sin}\left(H_m\right)\right] \end{gathered}$$ (6)

$$\begin{gathered} V_c=V_m+x_{10z}\mathit{sin}\left(V_m\right)-x_{12d}\mathit{cos}\left(2V_m\right)-x_{12c}\mathit{sin}\left(2V_m\right) \\ +k\left[-\frac{x_{1m}}{R_m}+\frac{x_2\mathit{cos}\left(V_m\right)}{R_m}+x_5+x_{10n}\mathit{cos}\left(V_m\right)\right] \end{gathered}$$ (7)

where R_c, H_c, and V_c are the corrected range, the corrected horizontal angle, and the corrected vertical angle respectively; R_m, H_m, and V_m are the measured range, the measured horizontal angle, and the measured vertical angle by the LT, respectively. The coefficient k is +1 for front-face and −1 for back-face measurements. The fifteen model parameters are described in Table 2. In this model, the horizontal angle H varies from 0⁰ to 360⁰, while the vertical angle V only ranges from 0⁰ to 90⁰.

Table 2:

Error parameters for LT with laser in head.

Error Name	Symbol	Description
Bird Bath Error	x 11	Calibration error in the distance to the birthplace of the reflector.
Transit Offset	x 2	Tilt axis displacement (T) with respect to azimuth (Z).
Scale error in the encoder	x12b, x12a	Second order errors of scale in horizontal and vertical encoders.
Beam Offset	x 1	Displacement of the point of emission of the laser beam with respect to the reference system origin. Is divided into two components:x1t and x1m which are determined projecting x1 on the tilt axis (T) of the LT and the axis normal to the tilting axis and the laser beam (M).
Beam tilt	x6	Inclination of the laser beam with respect to its nominal trajectory perpendicular to the tilt axis (T). x6 is decomposed into its projections on the tilt axis (x6t) and the beam perpendicular to the beam (x6m). The latter is not considered for having the same meaning as x5.
Transit Tilt	x 8	Error of perpendicularity between the tilt axis (T) and the azimuth(Z).
Encoder Eccentricity	x9x, x9y	Eccentricity errors of horizontal and vertical encoders. They are divided into components X, Y for the azimuthal (x9x) and (x9y) and z and n components (beam projection on the XY plane) for the tilt (x10n, x10z).
Scale error in the encoder	x12d, x12c	Second order errors of scale in horizontal and vertical encoders.
Vertical Offset Index	x 5	Zero offset vertically encoder.

The set of parameters to correct the measured values of an LT with a laser in the column is showed in the following equations:

$$R_c=R_m-x_{11}-2x_3\mathit{sin}\left(\frac{V_m}{2}\right)+k\left[x_2\mathit{sin}\left(V_m\right)\right]$$ (8)

$$\begin{gathered} H_c=H_m+\frac{1}{\mathit{sin}\left(V_m\right)}\left[\frac{-x_{1x}\mathit{cos}\left(H_m\right)+x_{1y}\mathit{sin}\left(H_m\right)}{R_m}+x_{6x}\mathit{cos}\left(H_m\right)-x_{6y}\mathit{sin}\left(H_m\right)-x_{12b}\mathit{cos}\left(2H_m\right)-x_{12a}\mathit{sin}\left(2H_m\right)\right] \\ +k\left[\frac{-x_{4t}}{\mathit{sin}\left(V_m\right)R_m}+\frac{x_7}{\mathit{cos}\left(\frac{V_m}{2}\right)}+x_8\mathit{tan}\left(\frac{V_m}{2}\right)+x_{9x}\mathit{cos}\left(H_m\right)-x_{9y}\mathit{sin}\left(H_m\right)\right] \end{gathered}$$ (9)

$$\begin{gathered} V_c=V_m+\frac{x_{1x}\mathit{sin}\left(H_m\right)+x_{1y}\mathit{cos}\left(H_m\right)-2x_3\mathit{cos}\left(\frac{V_m}{2}\right)}{R_m} \\ -x_{6x}\mathit{sin}\left(H_m\right)-x_{6y}\mathit{cos}\left(H_m\right)+x_{10z}\mathit{sin}\left(\frac{V_m}{2}\right)-x_{12d}\mathit{cos}\left(V_m\right)-x_{12c}\mathit{sin}\left(V_m\right) \\ +k\left[\frac{2x_2{\mathit{cos}}^2\left(\frac{V_m}{2}\right)-x_{4n}}{R_m}+x_5+x_{10n}\mathit{cos}\left(\frac{V_m}{2}\right)\right] \end{gathered}$$ (10)

where: R_c, H_c, and V_c are the corrected range, the corrected horizontal angle, and the corrected vertical angle respectively; R_m, H_m, and V_m are the measured range, the measured horizontal angle, and the measured vertical angle by the LT, respectively. The coefficient k is +1 for front-face and −1 for back-face measurements. The twenty model parameters are described in Table 3. In this model, the horizontal angle H varies from 0⁰ to 360⁰, while the vertical angle V only ranges from 0⁰ to 90⁰. For the LS method, we don’t evaluated uncertainty of error parameters because as we will show in the results section, this method is not the most convenient for LT compensation.

Table 3:

Error parameters for LT with laser in column.

Error Name	Symbol	Description
Bird Bath Error	x 11	Calibration error in the distance to the birthplace of the reflector.
Mirror Offset	x 3	Mirror plane displacement with respect to its nominal rotation center.
Transit Axis Offset	x 2	Displacement of the tilting axis with respect to the azimuth.
Beam Offset	x 1	Displacement of the laser beam with respect to the vertical axis into components X, Y x1x, x1y.
Beam Axis tilt	x 6	Laser beam tilt about the vertical axis Z into its components X, Y x6x, x6y.
Scale error in the encoder	x12b , x12a	Second order errors of scale in horizontal and vertical encoders.
Offset Plate Cover	x 4	Displacement of the laser beam with respect to the vertical axis due to refraction at the crystal output into its components X, Y x4t, x4n.
Mirror Tilt	x 7	Mirror tilt about the tilt axis T.
Transit Axis tilt	x 8	Error of perpendicularity between the tilt axis T and vertically Z.
Horizontal Encoder Eccentricity	x 9	Horizontal encoder eccentricity components X, Y x9x, x9y.
Vertical Encoder Eccentricity	x 10	Vertical encoder eccentricity components z, n x10z, x10n.
Scale error in the encoder	x12d, x12c	Second order errors of scale in horizontal and vertical encoders.
Vertical Offset Index	x 5	Error angular position (inclination 90°) Vertical encoder.

Nonlinear least square method (NLLS) for LT geometry error parameters evaluation.

In this method we use equation 11 to evaluate the LT geometry errors parameters placing them as part of the bundled adjustment for the minimization of residuals.

$$\mathrm{arg\; min}f\left(G,T_{\overrightarrow{t},R},{\overrightarrow{C}}_i\right)=\mathrm{arg\; min}\sum _{j=1}^{2\times m}\sum _{i=1}^n{\left({\overrightarrow{w}}_{i,j}·{\overrightarrow{ϵ}}_{i,j}\right)}^2$$ (11)

where: G is the error model parameter set of the LT. There are 15 model parameters for trackers with laser in the head and 20 parameters for trackers with laser in the column, see Tables 2 and 3. The rest of variables were already explained in equation 1.

In the NLLS method, we evaluate the uncertainty of the error parameters using Monte-Carlo simulation. For this, we make iterations of equation 11 at least 300 times. In each iteration, we add random noise to the ${\overrightarrow{P}}_{i,j}$ points, the noise in this method is sampled from a normal distribution, whose parameters are equal to those parameters evaluated by fitting a normal distribution to the residuals of the composite points (eq. 3). Finally the reported uncertainty is the standard deviation of the 300 values of each parameter found.

Once the error parameters are calculated, according to equations 5 to 10, we can compensate each LT variable (i.e., r, θ, and ϕ of all points from all LT positions), and then the network of targets can be compensated. If the error parameters have an impact on the measurements of the network, then these parameters must decrease the magnitude, offsets, or tendencies, of the f-b differences, otherwise, the differences would be similar as before, and we can proceed with the LT uncertainty. Improving f-b differences represents the validation of the evaluated geometry error parameters because if those parameters did not decrease any differences, tendencies or offsets, then they do not need to be applied. For the NLLS method the uncertainty of the parameters are the reference for validation. If the parameter’s uncertainties are neglected then, we can apply the compensation. If the network of targets is compensated then, we must re-evaluate the composite points using the corrected average points of the front and back measurement.

2.3.5. Step 5

As already explained, according with equation 3, we consider the standard deviation of the residuals (rather than the standard deviation about the mean, it is the square root of the sum of squares of residuals divide by n − 1) between composite points, ${\overrightarrow{C}}_i$ (corrected and uncorrected), and the rotated-translated points, ${\overrightarrow{P}}_i$, from each LT’s position, as the measure of LT uncertainty. With equation 3, we evaluate the uncertainty for each LT variable (r, θ, and ϕ), u(r), u(θ), and u(ϕ) and propagate them to each 3D measured point in the network. The propagation is done with Monte-Carlo simulation using 300 pseudo-random samples with zero mean, and sigma evaluated with equation 3 as input parameters for the simulation (Calkins [2] demonstrated that using 300 samples to propagate uncertainty gives similar results within 5% if the propagation is done with more than 300 samples). The propagation will result in a discrete cloud of 300 points for only one X, Y and Z or 3D measured point. Subsequently the uncertainty of each LT variable can be propagated to the distances between 3D points. Distance uncertainty, U(D_meas) in equation 12 and equation 14, is evaluated with the standard deviation of all the distances evaluated with the discrete cloud of 300 point of two points with an uncertainty already evaluated (it means two points with an already propagated LT uncertainty). Calkins [2] named the discrete cloud of point as the uncertainty field of a coordinate point. The Monte-Carlo simulation or propagation of LT uncertainty to a point is nothing more than a cycle of 300 iterations or more, where a random noise is added to the point of interest, $\overrightarrow{P}$, in each iteration, resulting in a cloud of 300 different points. The noise in this method is sampled from a normal distribution, whose parameters are equal to those parameters evaluated by fitting a normal distribution to the residuals of the composite points (eq. 3). In the end, the standard deviation of all the 300 cloud of points is the reported uncertainty for that point. Instead of Monte-Carlo simulation traditional and simple GUM method, [1] transforming spherical coordinates to Cartesian coordinates could be used.

The uncertainties for each LT variable, the 3D points in the network, and the distances between these 3D points, were evaluated using SA software, [5] and validated using an script. It is essential to mention that SA software can determine the uncertainty of each LT variable at each LT position (in our case, five LT positions and 3 variables for each LT give us 15 different LT variable). SA can evaluate the uncertainty of each point measured in each LT position using the appropriate uncertainties. We don’t use the particular uncertainty of each LT variable in each LT position. Instead, because SA can evaluate an overall (average) uncertainty for each LT variable, see equation 3, we use the overall uncertainty of each LT variable as the LT uncertainty for each LT, see Table 4 and Fig. 13.

Figure 13:

Average uncertainty of each NMI’s LT, letters B and D means the same LT brand. The values of this plot are in table 4, column with and without compensation LS. All evaluated values are k = 1.

2.3.6. Step 6

To validate LT uncertainty, we propagated it to the distance between composite points (corrected and uncorrected) as explained in STEP 5 and U(D_meas) is computed. Afterward, U(D_meas) is compared against the error (L_error, in equation 13) between the distances calculated with composite points minus the calibrated distances, equation 12, see [8]. It is important to mention that equation 12 is valid even if the LT still have systematic errors because the network method takes into account any factor that influences the measurement.

$$U\left(D_{means}\right)>=L_{error}$$ (12)

In our particular case, we use the 3D composite points 16 to 19, see Fig. 2, to get the error between the calculated distance (D_meas) from composite points 16 to 19 (3 distances) and the reference distances (D_ref) calibrated with LOS method. The evaluated error is showed in equation 13.

$$L_{error}=D_{means}-D_{ref}$$ (13)

Another validation of LT’s uncertainty was the normalized error (NE) evaluated with equation 14, see [9], where NE must be less or equal to 1.

$$NE=\frac{\left|L_{error}\right|}{\sqrt{U{\left(D_{means}\right)}^2+U{\left(D_{ref}\right)}^2}}<=1$$ (14)

where:

L_error is defined by equation 13;

U(D_meas) is the distance uncertainty evaluated using the uncertainties of 3D points considered in the distance evaluated. The 3D points uncertainties were evaluated using the propagation of the overall uncertainty of each LT variable to each 3D point with equation 3. Step 5 explains how to compute U(D_meas);

U(D_ref) is the uncertainty for the calibrated distance using the LOS method. LOS method can compute the uncertainty taking into account and ”only for convenience” the LT’s MPE for the range of the LT coming from LT data-sheet using equation 15. We mention ”only for convenience” because there must be an appropriate uncertainty balance around the LT interferometer.

$$U\left(LOS\right)=U\left(D_{ref}\right)=MPE\left(range\right)/\sqrt{3}$$ (15)

As a final check, we also determine whether the calculated uncertainty of each 3D point is similar to the standard deviation (repeatability) of the measured points (stddev graphics of the points in the network, Step 2 in workflow).

3. Results

3.1. NMIs experimental setup

NIST have LTs from three different manufacturers in their laboratory. They are identified as A, B, and C. The 19 targets and 5 positions experiment was separately performed using all three LTs. NIST setup is shown in Fig. 5, where most of the targets rest in tripods, aluminum structures, carbon fiber structures, and the rest in blocks on the floor. The temperature conditions at NIST laboratory are around 20 °C ±0.5 °C. Also, targets positions at NIST sometimes exceed the 200 mm tolerance of 3D position, but are close to those imposed in Table 1. It means NIST does not fit the nominal points, but the arrangement is otherwise similar to Fig. 2. NIST has 19 SMRs, and they used all of them for measuring, thus avoiding the need to move a single SMR to each target location.

Figure 5:

NIST network of targets setup, each number represents a target. Points 16 to 19 (references distances) rest on tripods.

Fig. 5 shows targets 16 to 19 placed at the same height. The distances between these points were measured by a LT aligned with the points using the LOS method, where only LT interferometer is used. This was the distance between points used as a reference to validate the LT uncertainty; this is described later in section 3.3.

INTI has only one LT in their laboratory. The 19 targets and five position experiment was done with that LT. INTI setup is shown in Fig. 6, where most of the targets rest in the wall, and the rest in blocks on the floor. The temperature conditions at INTI laboratory are around 20 °C ±0.5 °C. Target positions at INTI are within 200 mm of 3D position tolerance close to those imposed in Table 1. INTI has only 1 SMR, so they move that single SMR to each target nest.

Figure 6:

INTI network of targets setup, each number represents a target. Due to camera field of view, points 6, 9, 10, 11, 12, 13 and 15 are not in the picture, but were there as part of INTI setup. The calibrated distances (points 16 to 19) were evaluated with a laser tracer.

Fig. 6 shows targets 16 to 19 placed in an aluminum rail. The distances between these points were measured by a laser tracer aligned with the rail. The distance between points measured using the laser tracer were used as a reference to validate the LT uncertainty as described later in section 3.3.

CENAM has only one LT in their laboratory. The 19 targets and five position experiment was done with that LT. CENAM setup is shown in Fig. 7, where most of the targets rest on the wall, and the rest on tripods, carbon fiber, and aluminum structures. The temperature conditions at CENAM laboratory are around 20 °C ±0.2 °C. Targets positions at CENAM are within 200 mm 3D position of tolerance, close to those imposed in Table 1. CENAM has 6 SMRs, so they place five SMRs on targets and move the 6^th SMR to the remaining 14 target nests .

Figure 7:

CENAM network of targets setup, each number represents a target. Figure shows a zoom of target 5 with an SMR on the target and LT in position 5. Points 16 to 19 (reference distances) are part of a carbon fiber gauge.

Fig. 7 shows targets 16 to 19 placed at the same height on a carbon fiber bar. The distances between these points were measured by a LT aligned with the points using the LOS method, [4], where only LT interferometer is used. The distance obtained using the LOS LT measurement between points is used as a reference to validate the LT uncertainty, as described in Section 3.3. Targets 7 and 13 are part of a reference Invar ball bar. CENAM also uses this bar to validate LT uncertainty.

INMETRO does not have any LT, so they move to DCTA/IAE (Departamento de Ciencia y Tecnologia Aeroespacial/Instituo de Aeronautica y Espacio) facilities to access an LT. DCTA has only one LT in their laboratory. The 19 targets and five position experiment was done with this LT. INMETRO-DCTA setup is shown in Fig. 8, where most of the targets rest in the wall, and the rest in tripods and blocks on the floor. The temperature conditions at DCTA laboratory are around 20 °C ±1 °C. Targets positions at DCTA are within 200 mm 3D position of tolerance close to those imposed in Table 1. DCTA has only 1 SMR, so they moved the single SMR to 19 target nests. Fig. 8 shows targets 16 to 19 placed at the same height on an steel bar. The distances between these points were measured by a laser tracker aligned with the points using the LOS method, [4], where only LT interferometer is used. Targets 7 and 13 are part of a reference Invar ball bar. The length of the bar was measured in INMETRO’s CMM that belong to the dimensional metrology laboratory.

Figure 8:

INMETRO-DCTA network of targets setup, each number represents a target.

INACAL has only one LT in their laboratory. The 19 targets and five position experiment was done with this LT. INACAL setup is shown in Fig. 9, where most of the targets rest on aluminum profiles, tripods, the wall, and the rest on blocks on the floor. The temperature conditions at INACAL laboratory are around 20 °C ±0.5 °C. Targets positions at INACAL are within 200 mm 3D position of tolerance close to those imposed in Table 1. INACAL has only 1 SMR, so they move that single SMR to each target nest. In the particular case of INACAL they have only one target, so they had to couple this target to a screw adapted at each position; see point 13 in Fig. 9. The target nests adaptor issue considerably increased the difficultly of the experiment, made it slow, and less accurate.

Figure 9:

INACAL network of targets setup, each number represents a target.

Fig. 9 shows targets 16 to 19 placed on an aluminum rail supported by tripods. The distances between these points were measured by a laser tracker aligned with the rail using the LOS method. The distance between points thus obtained was used as a reference to validate the LT uncertainty as described later in section 3.3.

3.1.1. Repeatibility test with each NMI setup

Fig. 10 and Fig. 11 show the results from a repeatability test for NMI’s laser trackers. The experiment was carried out only from LT position one measuring the targets 10 consecutive times from that position. In the NIST case, the test was done with 19 SMR; this avoids the movement of the targets when the SMR is set on the target nest.

Figure 10:

Repeatability (standard deviations of 10 measurements) LT NIST A, B, C.

Figure 11:

Repeatability (standard deviations of 10 measurements). All NMIs except NIST.

According to Fig. 10, no targets moved during measurement as all show similar repeatability. The repeatability values for H_m (Horizontal angle or angle around the LT vertical axis) and V_m (Vertical angle or angle around the LT horizontal axis) are around 0.020 mm, which looks similar for INTI and CENAM LTs, see Fig. 11 and the value for R_m (range of LT interferometer) is under 0.004 mm. The repeatability values for the angles H_m, and V_m were multiplied by the range R_m for all LTs so that all values are in the same units. NIST, CENAM and INTI used cyanoacrylate based epoxy to attenuate target drift; other targets are part of a carbon fiber gauge.

Fig. 11 shows the results from a repeatability test for INTI, CENAM, INMETRO and INCAL’s LT. In the case of the INTI setup, Point 15 shows a significant movement in range and angular coordinates. It is possible that this point moved prior to, or during the measurement of the network of points. Point 15 is not shown in Fig. 6, but it was placed on an unstable tripod and this is the reason of the poor repeatability. The repeatability of the remaining points are less or equal to 0.020 mm.

In the case of the CENAM setup, the repeatability of the points 11, 12, and 14 are higher than 0.020 mm for H_m, but the repeatability of the rest of the points are less or equal to 0.020 mm.

The test was done with 6 SMRs, so they move with the 6^th SMR to each nest target 14 times multiplied by 10 to do the test.

In the case of the INMETRO-DCTA setup, the repeatability of most of the points are less or equal to 0.035 mm. The test was done with 1 SMR, so they moved that SMR to each nest target 19, and repeated that process 10 times for this test.

INMETRO-DCTA used a glue-based on epoxy to attenuate target drift. From Fig. 11 we can see that the first five points, and the point 14 are moving taking into account only the LT range R_m.

In the case of the INACAL setup, point 15 shows a significantly large movement in the horizontal angle. It is possible, that this point moved prior to, or during the measurement of the network of points. The repeatability of the rest of the points are less or equal to 0.060 mm.

According to Fig. 11 most of the INACAL targets shows similar repeatability values but they are larger in comparison to those seen by other NMIs, probably because of the screw used to attach the target. INACAL doesn’t use glue; instead, as explained before, they screw the only target they have it every time they want to measure, and because of this, repeatability is rather poor.

3.1.2. Front minus back measurements

Figs. 12, shows the average and standard deviation differences between front-face and back-face measurements (f-b) for all NMIs LTs. The average and standard deviation were evaluated with the 95 measurements made by each LT of each NMI, that is, 19 X, Y, Z points measured from 5 different positions of the LT.

Figure 12:

NMIs Front-Back Measurements for all LTs without and with compensations.

According to Fig. 12 we can observe that all the LTs with the exception of the LT of CENAM and INACAL present considerable differences (f-b) in the original average of X, Y, Z coordinates. This means that the remaining NMIs did not perform an adequate compensation of their respective LTs as indicated in step 1 of the flow diagram of Fig. 3. This is something very common in LTs that are constantly moving from one place to another under different conditions. In the case of CENAM, the LT was compensated prior to the measurements, following an exhaustive procedure (full compensation). In the case of INACAL, its LT had just been acquired and it came with full compensation. The LT of INTI received an intermediate compensation but due to time it was not possible to make a full compensation. For the rest of the LTs of the NMIs, step 1 of the flow diagram of Fig. 3 was ignored and the LTs were used as they were.

In Fig. 12 we can see that the compensation with the LS method performs well especially on the Y-axis if we only consider the average of the f-b measurements. However, if we take into account the standard deviation of the f-b measurements, we can see that the improvement is not so clear except for the case of the NIST LT C. In the same Fig., we can see that the NLLS method is just as efficient in the average of the X, Y, Z coordinates of the f-b measurements as the LS method, especially in the Z-axis. However, if we observe the standard deviation in the X, Y, Z axes of the f-b measurements, in the NLLS method we can see values closer to zero in all the axes and in all the LTs except in the X, Y axes of the LT B of NIST and the LT of INMETRO. The above leads us to conclude that the NLLS method is the most appropriate to compensate a LT using the network of points method.

Based on Fig. 12 we should not apply any compensation for CENAM and INACAL LTs to validate LT uncertainty, but we apply it anyway. This will be noted in section 3.3.

3.1.3. Composite points

The composite points before compensation were used as the reference to determine LT error model parameters. After obtaining the error model parameters, the front-face and back-face data measured by the LTs from the five positions were corrected based on the determined model parameters. A bundle-adjustment process was then performed again to determine the composite points after compensation. These are considered as the final reference coordinates of the network.

3.2. LT uncertainty comparison

Table 4 compares the uncertainty for all LTs from the different NMIs. This table shows that the average uncertainties for NIST LTs are about half of the other laboratories. This is easily explained because NIST has 19 SMRs, so they did not have to move a single SMR through the network. Also, the structures and tripods used by NIST were very stable. Besides, average uncertainties between NIST’s three LTs are almost the same, which indicates that the LTs were well compensated with the LS and NLLS methods, and uncertainty under the same conditions is around the same. Table 4 also shows that the uncertainties between the rest of the laboratories with INACAL’s exception are comparable under the same conditions and the same setup of the network of points. INTI’s LT uncertainties are a somewhat smaller because most of the targets in their setup were fixed to stable walls, not so in the case of CENAM and INMETRO-DCTA. In the case of CENAM, some targets were glued to walls lined with a double aluminum sandwich, and in the case of INMETRO-DCTA, they use an epoxy glue that does not hold fix the targets. In the case of INACAL, we must mention that the lack of targets affected the measurements. This became evident when the parametric errors of their LT could not be evaluated. In fact, the repeatability shown in Fig. 11 indicates that mounting a single target threaded onto the screws is not the most appropriate. INACAL’s data is very valuable because it shows that the network method works because the LT uncertainty increases, see Fig. 13, not by the LT itself but by the setup of the points.

Due to the aforementioned, Table 4 and Fig. 13 shows that the network method performs well, and the evaluated LT uncertainty using this method is representative of the different setup circumstances, with similar environmental conditions, similar volume, and a similar arrangement of points.

We must mention that Table 4 and Fig. 13 don’t show values that reflect the LT accuracy of different manufacturers. Instead, they indicate that the network method can be used for LT uncertainty of a particular environment, setup, operator, another factors, etc. Also, the network method can be used for performance evaluation of a coordinate measuring system like an LT, a laser radar, a CMM, etc. and the evaluation of its geometric errors. Here, we don’t explore the application for performance evaluation of an LT.

Fig. 13 shows the average uncertainty for each NMI’s LT coming from Table 4. The evaluated uncertainties of Table 4 and Fig. 13 were calculated using SA software and a script to validate SA calculations, [5]. This software allows outlier’s removal (i.e. remove points that moves during the measurement using a specific criteria). Table 5 shows the points removed at each NMI’s network. The removal uses the ”Autosolve” option in SA. This option was selected until all points were under 100 percent according to SA’s criteria. It is important to mention that SA uses a weighted uncertainty for the three LT variables. In this case, we consider the LT MPEs from LT data sheet as the starting weights. Also, SA has some extra weights for each variable, which we leave at default values of 1, it means no additional weights were used. Fig. 13 shows only the uncertainties of Table 4 evaluated with LS compensation method. Uncertainties evaluated with the NLLS were not plotted because with the exception of INMETRO LT, these are very similar to LS compensation.

Table 5:

Removed points for each LT. N.E. means not evaluated.

Lab	LT type	Removed points LS	LT pos	Removed points NLLS	LT pos
NIST	A	P5	5	P5	5
		P6	5	P4	5
		P16	5	P14	4
	B1	P12	1 y 2	P5	4
				P15	3
				P1	1 y 2
				P7	1
	C	P3	3	P3	3
		P7	4	P15	1
		P15	1	P6	4
INTI	D1	P8	5	P9	5
		P9	5	P11	5
		P11	5	P8	5
		P12	5	P12	5
		P15	4	P5	5
		P6	5	P4	5
		P10	5
CENAM	B2	P5	3,4 y 5	P12	4
		P11	4 y 5	P8	4
		P12	4 y 5	P5	3, 4 y 5
INMETRO/DCTA	B3	P6	1 y 2	P15	4
		P12	3,4 y5	P5	Co
		P14	4	P7	4
		P15	4	P14	4
		P16	1,2,4 y5	P13	4
		P17	1 y 2	P16	1,2 y 5
		P18	1,2,3 y 5	P8	3,4 y 5
		P19	1,2,3,4	P18	1 y 2
				P17	1 y 2
				P6	1,4 y 5
INACAL	D2	P2	3	N.E.	N.E.
		P3	3	N.E.	N.E.
		P4	2	N.E.	N.E.
		P5	1,2,4 y 5	N.E.	N.E.
		P8	1 y 3	N.E.	N.E.
		P9	1 y 2	N.E.	N.E.
		P10	4 y 5	N.E.	N.E.
		P11	4 y 5	N.E.	N.E.
		P12	3	N.E.	N.E.
		P16	all	N.E.	N.E.
		P17	1,2,3 y 4	N.E.	N.E.
		P18	1,2,3 y 4	N.E.	N.E.
		P19	1,2,3 y 5	N.E.	N.E.

3.3. LT uncertainty validation

As shown in the last step of the workflow of Fig. 3 if the uncertainty of the distance evaluated with the network method using a LT is greater than the distance’s error resulting from the measurement of a distance calculated with composite points minus their respective reference distance, then the uncertainty is validated, see [8], lets call this criterion U(CD) > E. The problem with this statement is, how significant should be the uncertainty?. Due to this, another criterion used to validate uncertainty was the normalized error (NE), lets call this criterion NE < 1, see [9] and finally, the standard deviations of the points criterion. The normalized error parameter is shown in Fig. 15. This error is computed with the distance errors and the expanded uncertainties (k = 2) given by equation 14.

Figure 15:

Normalized errors of 3 inter-target distances in the 19-target network for corrected and uncorrected composite distances errors. In figure (b) the NE values for CENAM LT must be multiplied by 10

For the case of uncertainty validation with the U(CD) > E or NE < 1 criteria, we could use as many distances as possible evaluated with all combinations of the network of points, but the problem is the time and effort to establish the calibrated distances values. Due to this, we use only three distances, that we will call composite distances because they were evaluated with the composite points 16 to 19 with LT compensation and without LT compensation, and represent our measurement, D_meas in equation 13. The unknown true or reference value of each distance, D_ref in equation 13 were measured by the LOS method, [4]. These LOS distances are considered as the reference with tracebility for this validation process. With D_meas and D_ref, the distances errors between the composite distances and the LOS distances are computed according with equation 13.

Now that we have the distance error, we must evaluate the uncertainty of the composite distances (U(CD) in last step of workflow of Fig. 3) so that we can accomplish the test of uncertainty validation per equation 12 U(CD) > E criterion. The process for U(CD) evaluation is: table 4 and Fig. 13 show the LT uncertainty that we evaluated through the bundle adjustment process. Then, we use SA software and our own script (which uses Montecarlo simulation) to propagate the LT uncertainty to the composite points in the network. Afterward, we evaluate the uncertainty in the 3 distances between points 16 to 19 (i.e. 16 to 17, 16 to 18, and 16 to 19). We refer to these uncertainties as U(D_meas) in equation 12 that are the same as U(CD) in workflow of Fig. 3. These uncertainties were then compared with the distances errors; Fig. 14 shows this validation.

Figure 14:

${\overrightarrow{U}}_{LT}$, LT uncertainty validation using the criteria U(D_meas) > error. In figure (b) the error values for CENAM LT must be multiplied by 10. All evaluated values are k = 2.

For evaluation of NE and the use of NE < 1 criterion, the only remaining term we need is the uncertainty in the reference U(D_Ref), which was evaluated, as explained in [4] using equation 15. Fig. 15 show the validation using the NE < 1 criterion.

In Fig. 14, we can see that U(D_meas) is bigger than the error in most of the cases, and in Fig. 15, the normalized error (NE) is below one in most of the cases which validates the LT uncertainty. However, for CENAM’s LT, the performed LS compensation shows that the LT gets worse, U(D_meas) is not more significant than the error, and NE is beyond one (look at the label of Figs. 14(b) and 15(b) where the error values and NE must be multiplied by 10). This is evident in Fig. 12, where we can see that CENAM’s LT doesn’t have any tendency in front-face minus back-face measurements, but we did the compensation. This shows the possibility that the LS method to find LT geometrical errors will decrease the LT accuracy if it doesn’t need it. Because of this, the protocol of workflow in Fig. 3 mentions that we must compensate the LT just in the case the front-face minus back-face measurements have an offset or tendency. For NLLS compensation CENAM’s LT did not get worse, see Fig. 14(c) and 15(c). It means that we can employ NLLS method for compensation.

The other case where U(D_meas) is not larger than the error is the distance between points 16 and 17 for NIST LT A. In this case, however, NE still smaller than 1, see Fig. 15. This case can be explained as some movement of the points during measurements not taken into account for the outliers evaluated using SA software, Table 5.

As shown in Fig. 15, the NE (i.e., the normalized error) for the 3 distances for all NMIs are all smaller than one without and with compensation. This result shows that the errors of the composite and corrected composite distances are smaller than our estimate of the expanded uncertainties, which at the (k = 2) level have about a 95 % level of confidence (the composite distances and corrected composite disatnces are the distances evaluate with the composite points before and after LT geometrical error compensation). The claimed uncertainties are consistent with our assumption that these are greater than the evaluated error and below for the normalized error; then, the evaluated uncertainty using the network method is validated. According with Fig. 15 there are some cases where the uncertainties are over estimated or the errors are too small. Fig. 14 shows that we are in the last case where the errors are small, and the evaluated uncertainties are consistent.

Finally for the standard deviation criterion, Fig. 14 shows that the uncertainty values of U(D_meas) are around the standard deviation of the points in the network, compare Fig. 14 with Figs. 10, and 11. This metric is just to see that the evaluated uncertainty must be comparable with the standard deviation of the points and not significantly larger or smaller than these values.

Notice that INMETRO-DCTA and INACAL uncertainty validation is not part of Figs. 14, 15; this is because points 16 to 19 were removed from the network of points, see Table 5. The reason was that INMETRO-DCTA and INACAL use an unstable bar to realize those points, and the points were not stable during the measurement.

4. Discussions and conclusions

In this article, we have shown that the network method allows evaluating the LT uncertainty in real environments. We have validated this method, by propagating the LT uncertainty to the network of points and later to distances between points and showing that this distance uncertainty is greater than the error using a calibrated distance evaluated with the LOS method. The LOS method gives us the advantage that we do not have to have a calibrated bar for LT uncertainty validation; then, there is no need to travel with any gauge to the place where the measurements for LT uncertainty validation were performed. The traceability of the LOS method can be achieved calibrating in frequency the LT interferometer. We also have used the NE as a metric to prove that LT uncertainty that we have evaluated is reasonable (i.e., not conservative). Something to remark, which is not appreciated in Fig. 15 is that the distance error increases slightly in most cases when the LT is compensated with the LS method and in a few cases when the NLLS method is employed. However, NE also increases; this addresses the balance that must exist between the error and the evaluated uncertainty and shows that the LS and NLLS methods improve not only LT measurement but LT uncertainty, see Table 4. Table 4 shows that sometimes LT uncertainty increases with the compensation, and sometimes it decreases. Our conclusion is that we must apply for compensation only when the evaluated parameter’s uncertainties are not similar to the evaluated parameter’s values. Also, we must apply LS or NLLS compensations when there is a clear offset or tendency between f-b differences in all LT positions.

In this article, we have shown two methods for LT compensation, the least square method (LS), see equation 4, and the nonlinear least square method (NLLS), see equation 11. The LS method that uses the composite points as a reference can improve LT measurements because, in most of the cases, offsets between front-face minus back-face measurements (f-b) decrease, so this validates the evaluated LT geometrical error. However, the LS method did not improve all parameters because after compensation, some parameter’s values still the same. The NLLS method shows a better performance than the LS method because all parameter’s values were zero after compensation. Besides, with the NLLS method, we have evaluated the uncertainty of the evaluated parameters. This uncertainty is an indicator of LT compensation; if the uncertainties are almost zero or not close to the evaluated parameter’s values, then we can apply for the compensation.

Documentary standards like [13], [14], and [15], mention that f-b is a simple method to identify poor LT performance, but these standards don’t describe how to decrease those differences. The LS and NLLS methods described here shows us how to reduce those differences. Muralikrishnan et al., [19], had already studied the errors that are sensitive to the f-b measurements, but they, like documentary standards, don’t describe how to improve the f-b measurements.

Finally, we conclude that we have validated the network method as a dynamic procedure for LT uncertainty evaluation (look the initial LT uncertainties and those evaluated for a particular setup in Table 4) and identify a poor LT performance when we evaluate the geometric errors using the LS and NLLS methods. The network method for LT uncertainty was validated using calibrated distances, and under similar environmental factors, actually laboratory environments with temperatures of 20 °C ± 1°C and for LT’s geometrical errors with improvements in f-b measurements. The LT uncertainty evaluated with the network method is simple, can be evaluated in real-time, and can be expanded to subsequent measurements of the work-piece of interest to be measured. The LT uncertainties evaluated with the network method can be propagated to points, distances, angles, diameters, etc., using Monte Carlo simulation and an uncertainty represented by a cloud of points. The input parameters for that simulation were 300 pseudo-random samples, with mean zero, and sigma evaluated with equation 3. The network method described here uses of the MPE’s instruments specified by the manufacturer in a data-sheet reflecting in this way the quality of the data points delivered by the LT and then the reported uncertainty. Afterward, the LT performance evaluation that primary or secondary laboratories realize in all the world charges more value because, based on this evaluation, a new MPE can be established. In this work, the LT’s MPE was used as initial weights for the residuals of equation 1 that are the base for LT uncertainty evaluation. For a better reference of the MPE, see the standards [13], [14], and [15].

We must say that the great disadvantage of the network method is that it evaluates an uncertainty that includes all the factors that influence the uncertainty. This does not allow us to distinguish, which is the factor that is generating uncertainty in the measurements to reduce or remove it, but at least we know that something is wrong. Because of this, we add the LT geometry error parameters evaluation to remove at least the systematic errors. As discussed at the beginning, the network method can be used so that the industry can finally report uncertainty in a practical way and identify if something is wrong with its measurements or its LT using the evaluated error parameter.

In the future we are planning to include more and diverse calibrated distances for LT uncertainty validation. Also, we are planning to compare the network method and another known method for dimensional measurement uncertainty of large parts. Also, the network method must be validated under different environmental factors to see if it still valid. Finally, we are going to study the influence of the number of points and LT positions in the estimated uncertainty. We choose five LT position because normally this value is statistically sufficient to measure something or even cover the perimeter of a large parts using only an LT.