Algorithm 1 Object Measurement

$Z_{picker}$: The lowest position of the picker from the lens in z-axis

$D_{obj}$: The maximum depth of objects

$rows$, $cols$: The rows and columns in the 2D pixel coordinate system

$d(rows,cols)$: The depth value at $(rows,cols)$

$A_{min}$: The minimum area of object cross section

C: The contour of the object

$w,h,d,\varphi$: The width, height, depth and orientation of the object

$W,H$: The width and height of the depth frame

$(p_x,p_y)$: The picked position in $xy$-axes on the object

(1) Acquire sensor data.

Capture a depth frame from D435

if the depth frame is not successfully captured then

return the error code of ’$-1$’

end if

Calculate point clouds from the depth frame

(2) Extract object area.

Separate the pixels where $Z_{picker}-D_{obj}<d(r,c)<Z_{picker}$

Find the contours of the separated pixels whose area is larger than $A_{min}$

if no contour is found then

return the error code of ’$-2$’

end if

Select the largest contour as C

if C is out of the depth frame bound then

return the error code of ’$-3$’

end if

(3) Calculate d, w, h and $\varphi$.

Accumulate the point clouds in C

$z_{avg}$ = average of the z-axis values of the point clouds

d = $Z_{picker}-z_{avg}$

Find a rotated rectangle of the minimum area enclosing C

Calculate the scaling factor S from the point clouds

w = width pixels of the rectangle $\times S$

h = height pixels of the rectangle $\times S$

$\varphi$ = rotation of the rectangle

(4) Calculate $p_x$ and $p_y$.

$(c_x,c_y)$ = average of the four corner positions of the rectangle in pixels

$(d_x,d_y)$ = ($c_x$ − $W/2$, $H/2$ − $c_y$)

$\left[\begin{array}{c}{p}_x\\ p_y\end{array}\right]=-S\left[\begin{array}{l}cos\left(\varphi \right) -sin\left(\varphi \right)\\ sin\left(\varphi \right) cos\left(\varphi \right)\end{array}\right]\left[\begin{array}{c}{d}_x\\ d_y\end{array}\right]$

(5) Return w, h, d, $\varphi$, $p_x$ and $p_y$