Uncalibrated Visual Servoing for Underwater Vehicle Manipulator Systems with an Eye in Hand Configuration Camera

Abstract

This paper presents an uncalibrated visual servoing scheme for underwater vehicle manipulator systems (UVMSs) with an eye-in-hand camera under uncertainties. These uncertainties contain vision sensor parameters, UVMS kinematics and feature position information. At first, a linear separation approach is addressed to collect these uncertainties into vectors, and this approach can also be utilized in other free-floating based manipulator systems. Secondly, a novel nonlinear adaptive controller is proposed to achieve image error convergence by estimating these vectors, the gradient projection method is utilized to optimize the restoring moments. Thirdly, a high order disturbance observer is addressed to deal with time-varying disturbances, and the convergence of the image errors is proved under the Lyapunov theory. Finally, in order to illustrate the effectiveness of the proposed method, numerical simulations based on a 9 degrees of freedom (DOFs) UVMS with an eye-in-hand camera are conducted. In simulations, the UVMS is expected to track a circle trajectory on the image plane, meanwhile, time-varying disturbances are exerted on the system. The proposed scheme can achieve accurate and smooth tracking results during simulations.

1. Introduction

Nowadays, underwater vehicles play a pivotal role in the applications of oceanic explorations, submarine salvages and scientific expeditions [1,2]. Highly integrated underwater vehicles, e.g., Ocean One [3], SAUVIM I Intervention Autonomous Underwater Vehicle (I-AUV) [4], and Girona 500 I-AUV [5] are developed for the field intervention operations based on its camera-robot systems. Compared with remote operations of underwater vehicles, autonomous underwater operations by visual servoing not only reduce the dependence on the mother ship, but also decrease disturbances from the umbilical cables, which makes underwater vehicles complete manipulations autonomously with less operations from professional pilots [6,7].

Visual servoing based on the vision sensor (i.e., underwater camera) have gained significant attentions in underwater autonomous interventions. The purpose of visual servoing on underwater vehicles is to control the end-effector to complete the specific intervention tasks through visual information [8]. The visual servoing can be generously divided into two categories [9], the position-based visual servoing (PBVS) and the image-based visual servoing (IBVS). The PBVS scheme demands visual sensors to obtain three-dimensional (3 D) position of the target and then control the end-effector approach to the desired position, which means accurate calibration results of the camera-robot system are often indispensable [10]. Classical IBVS scheme guides the end-effector directly through the information of the target, its core skill is to calculate the inverse or the transpose of the image Jacobian matrix (interaction matrix) [11]. However, these mentioned schemes require intrinsic and extrinsic parameters of the camera. It is well known that camera calibrations are very costly and tedious, especially in underwater environment [12]. Therefore, in order to achieve satisfied visual servoing results in underwater interventions, it is not suitable to directly utilize either the PBVS scheme or the classical IBVS scheme.

To avoid the influences of inaccurate camera calibrations, without perfectly knowing actual parameters of the vision sensor, uncalibrated visual servoing based on the IBVS scheme has attracted great attentions in last twenty years. A large portion of works turned image Jacobian matrix calculation to a parameter identification problem by establishing proper cost functions [13]. Then, without knowing the exact knowledge of camera parameters in advance, the image Jacobian matrix in the IBVS scheme can be acquired through estimating its elements by numerical estimations (i.e., Broyden-based method [14], recursive least square method [15], Kalman method [16], SVM based method [17] etc.). However, such kinematic-based uncalibrated visual servoing schemes usually ignore the nonlinear forces in robot dynamics and assumed the controller of the system is ideal, further, they do not take control schemes into stability analysis.

Such drawbacks are largely remedied under dynamic based visual servoing schemes by using a depth-independent interaction matrix [18], this pioneering work proposed a method to deal with the case of time-varying and uncertain depth in visual servoing process. Wang et al. [19] investigated uncalibrated visual servoing problem with uncertain models, they presented the passivity properties in the overall kinematic system. In [20], Wang H. et al. proposed an adaptive controller with a velocity observer to avoid velocity measurement noises. For object grasping, a manipulator with an eye-in-hand camera is common, because the feature points should be extracted on the object, however, these articles only concern the situation about the eye-to-hand camera configuration. Liang et al. [21] proposed a unified method for both eye-in-hand and eye-to-hand camera configurations, this work also gave a separation algorithm on kinematic properties of the manipulator system, however, the uncertain kinematics of the manipulator was not considered. Wang et al. [22] proposed a dynamic uncalibrated visual servoing approach for a free-floating based manipulator, however, it assumed the system has no external disturbances, and this assumption is often hard to satisfy in underwater environment.

Although some researches that involve uncalibrated visual servoing have been launched on industrial manipulators and nonholonomic mobile robots, visual servoing for the UVMS is a largely under explored domain. Li et al. [23] presented a learning-based IBVS scheme for an absorptive underwater vehicle to realize autonomously sea-cucumbers capture. Gao et al. [24] proposed a hierarchical IBVS strategy for the underwater vehicle dynamic positioning problem. Sivčev et al. [25] proposed an engineering oriented PBVS scheme for work class hydraulic ROVs. The drawback with these strategies is, they require the accurate camera parameters in advance. In the underwater manipulation domain, there are some examples of researches on uncalibrated visual servoing, for example, Xu et al. [26] investigated a dynamic based uncalibrated visual servoing scheme for the underwater soft robot. However, to the best of our knowledge, there is limited work on dynamic based uncalibrated visual servoing concerned with the UVMS. Unlike industrial manipulators or mobile robots, the visual servoing process of the UVMS suffers from two aspects: on one hand, since the UVMS is a free-floating multibody mechanism, the inner coupling effects between the manipulator and the vehicle are strong and cannot be neglected [27]. On the other hand, the underwater environment is complex, which makes the system tend to be affected by currents, accurate calibration results of the camera-UVMS system are hard to achieve in underwater environment. These notable differences from other types of robots make it improper to directly employ previous visual servoing studies on UVMSs.

To solve these problems, we propose a novel uncalibrated visual servoing method for UVMSs with an eye-in-hand configuration camera. The main contributions of this work can be summarized as follows: firstly, we propose a novel linear separation method to collect some uncertainties into constant vectors compared with the one in [21,28]. This method can be applied for other free-floating based articulated manipulators. Secondly, to optimize the restoring moments of the UVMS, a new reference velocity term is proposed through exploiting the kinematic redundancy of the UVMS, moreover, without perfectly knowing the camera parameters, kinematics of the UVMS and the target position, a novel composite controller with new adaptive laws is proposed to solve the visual servoing problem. Thirdly, considering time-varying disturbances of the UVMSs, a high order disturbance observer is presented to estimate and compensate such disturbances, and stability analysis is given under the Lyapunov theory.

The rest of this paper is organized, as follows: in Section 2, we introduce the kinematic relationship of the system in UVMS, the dynamic model of the UVMS is also established. Thereafter, a novel adaptive controller with a disturbance observer is proposed for uncalibrated visual servoing in Section 3, stability proof analysis is also given is this section. In Section 4, numerical simulations on a 9-DOFs UVMS are carried out to demonstrate the effectiveness of the proposed scheme. The flow chart of this work is given in Figure 1.

Figure 1.

Flow chart of this work.

Notation: Throughout this paper, we define ${\epsilon }_{a,b}^c$ as the vector from the frame $a$ to the frame $b$ expressed in the frame $c$. $R_a^b$ and $T_a^b$ are the rotation matrix and the homogeneous transformation matrix from the frame $a$ to the frame $b$ respectively. Moreover, frames $\mathrm{I},\mathrm{V},\mathrm{ee},\mathrm{fe},\mathrm{cam}$ represent the inertial frame, the vehicle fixed frame, the end-effector frame, the target frame, and the camera frame respectively, frame $i\in \left\{0,1,2\dots n\right\}$ represents the frame of the $i$-th joint.

2. Kinematics and Dynamics

In underwater visual servoing, camera parameters, UVMS kinematic modelling, and feature position are always difficult to accurately achieve. A linear separation method to collect camera uncertainties into vectors is given in [18,21], however this method can only be used for the fixed manipulator and under the assumption that kinematics of the manipulator is perfectly known. In this section, we propose a novel linear separation method to collect these uncertainties into constant vectors without this assumption for free-floating based articulated manipulators.

2.1. Kinematics

In this paper, we consider a $6+n$ DOFs UVMS with a standard fixed pinhole camera, the camera is fixed on the end-effector. A target is fixed w.r.t. to the inertial frame, and the feature point is extracted from the target. The image coordinates $x={[u,v]}^T$ of the feature point can be written as [18]:

$$\left[\begin{array}{c}x\\ 1\end{array}\right]=\frac{1}{z(\zeta )}{\mathrm{MT}}_{\mathrm{I}}^{\mathrm{ee}}\left[\begin{array}{c}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}\\ 1\end{array}\right]$$ (1)

where $z(\zeta )$ is the depth of the feature in the camera frame. $\mathrm{M}=\mathsf{\Phi }{\mathrm{T}}_{\mathrm{ee}}^{\mathrm{cam}}$, $\mathrm{M}\in {ℝ}^{3\times 4}$ is the perspective projection matrix, which contains intrinsic and extrinsic parameters of the camera. $\mathsf{\Phi }\in {ℝ}^{3\times 4}$ depends on intrinsic parameters of the camera and ${\mathrm{T}}_{\mathrm{ee}}^{\mathrm{cam}}$ depends on extrinsic parameters of the camera. Since the camera is fixed on the end-effector, ${\mathrm{T}}_{\mathrm{ee}}^{\mathrm{cam}}$ is time-invariant. Therefore, the perspective projection matrix $\mathrm{M}$ can be concluded to be a constant. Define ${\mathrm{M}=[\mathrm{m}}_1{,\mathrm{m}}_2{,\mathrm{m}}_3{]}^{\mathrm{T}}$, ${\mathrm{m}}_i\in {ℝ}^{1\times 4}$ is the i-th column of $\mathrm{M}$. Depth $z(\zeta )$ can be obtained as:

$$z(\zeta )={\mathrm{m}}_3^T{\mathrm{T}}_{\mathrm{I}}^{\mathrm{ee}}\left[\begin{array}{c}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}\\ 1\end{array}\right]$$ (2)

consider a UVMS with a n-DOFs manipulator, define $\dot{\zeta }{=[\mathrm{v}}_v^T,{\omega }_v^T,{\dot{q}}^T{]}^T\in {ℝ}^{(6+\mathrm{n})\times 1}$ as the generalized velocity term of the UVMS, where ${\mathrm{v}}_v^{}\in {ℝ}^{3\times 1}$ and ${\omega }_v^{}\in {ℝ}^{3\times 1}$ denote the linear and angular velocity of the vehicle in the vehicle fixed frame. Let ${\dot{\eta }}_v{=[\mathrm{v}}_v^T,{\omega }_v^T{]}^T$ denote the generalized velocity of the vehicle. $q\in {ℝ}^{n\times 1}$ is the vector represents joints position of the manipulator. By differentiating (1), the relationship of $\dot{\zeta }$ and $\dot{x}$ can be expressed as:

$$\dot{x}=\frac{1}{z(\zeta )}J(\zeta ,x)\dot{\zeta }$$ (3)

where $J(\zeta ,x)=\left[\begin{array}{c}{\overline{\mathrm{m}}}_1^T-u{\overline{\mathrm{m}}}_3^T\\ {\overline{\mathrm{m}}}_2^T-v{\overline{\mathrm{m}}}_3^T\end{array}\right]\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}+t)$ is the depth- independent Jacobian matrix, $R_{\mathrm{I}}^{\mathrm{ee}}$ and $t$ are the rotation and translation of ${\mathrm{T}}_{\mathrm{I}}^{\mathrm{ee}}$. And it can be further divided into two parts [20]:

$$J(\zeta ,x)=J_{\mathrm{zin}}(\zeta )-x{J}_{\mathrm{zd}}(\zeta )$$ (4)

On one hand, $J_{\mathrm{zin}}(\zeta )$ is so called depth-rate-independent Jacobian matrix, which maps the generalized velocity term $\dot{\zeta }$ to a plane parallel to the image plane. $J_{\mathrm{zin}}(\zeta )$ can be represented as:

$$J_{\mathrm{zin}}(\zeta )=\left[\begin{array}{c}{\overline{\mathrm{m}}}_1^T\\ {\overline{\mathrm{m}}}_2^T\end{array}\right]\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}+t)$$ (5)

${\overline{\mathrm{m}}}_{\mathrm{i}}$ is the first three elements of ${\mathrm{m}}_{\mathrm{i}}^{}$.

On the other hand, $J_{\mathrm{zd}}(\zeta )$ is so called depth-rate-dependent Jacobian matrix, which describes the relationship between $\dot{\zeta }$ and $\dot{z}(\zeta )$. $J_{\mathrm{zd}}(\zeta )$ and $\dot{z}(\zeta )$ can be represented as:

$$J_{\mathrm{zd}}(\zeta )={\overline{\mathrm{m}}}_3^T\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}+t)$$ (6)

$$\dot{z}(\zeta )={\overline{\mathrm{m}}}_3^T\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}+t)\dot{\zeta }=J_{\mathrm{zd}}(\zeta )\dot{\zeta }$$ (7)

The overall kinematics (2), (5), (7) share following properties.

Property 1.

For a vector$\eta \in {ℝ}^{(6+\mathrm{n})\times 1}$,$J_{\mathrm{zin}}(\zeta )\eta$can be represented as the product of a regressor matrix$Y_{\mathrm{zin}}(\zeta ,\eta )$and a constant vector$a_{\mathrm{zin}}$.

$$J_{\mathrm{zin}}(\zeta )\eta =Y_{\mathrm{zin}}(\zeta ,\eta )a_{\mathrm{zin}}$$ (8)

Proof. 

First, we will prove that

$$\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}+t)=\frac{\partial }{\partial \zeta }({\epsilon }_{fe,\mathrm{I}}^{\mathrm{ee}}-{\epsilon }_{\mathrm{ee},\mathrm{I}}^{\mathrm{ee}})=-\frac{\partial }{\partial \zeta }(R_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{I},\mathrm{ee}}^{\mathrm{I}})=-(R_{\mathrm{I}}^{\mathrm{ee}}\frac{\partial {\epsilon }_{\mathrm{I},\mathrm{ee}}^{\mathrm{I}}}{\partial \zeta }+\mathsf{\Upsilon }{\epsilon }_{\mathrm{I},\mathrm{ee}}^{\mathrm{I}})=Y_{\alpha }(\zeta ,\eta )a_{\alpha }$$ (9)

where $\mathsf{\Upsilon }=\left[\begin{array}{c}\frac{\partial (r_{11}+r_{21}+r_{31})}{\partial \zeta }\\ \frac{\partial (r_{12}+r_{22}+r_{32})}{\partial \zeta }\\ \frac{\partial (r_{13}+r_{23}+r_{33})}{\partial \zeta }\end{array}\right]$, and $r_{ij}$ is the (i,j) element of $R_{\mathrm{I}}^{\mathrm{ee}}$.

On one side, it can be found that:

$$\begin{array}{l}\frac{\partial {\epsilon }_{\mathrm{I},\mathrm{ee}}^{\mathrm{I}}}{\partial \zeta }\eta =[\begin{array}{ccc}{R}_{\mathrm{V}}^{\mathrm{I}}& -(S(R_{\mathrm{V}}^{\mathrm{I}}{\epsilon }_{\mathrm{V}0}^{\mathrm{V}})+S(R_0^{\mathrm{I}}{\epsilon }_{0,\mathrm{ee}}^0))R_{\mathrm{V}}^{\mathrm{I}}& J_{pos,man}\end{array}]\eta \\ =R_{\mathrm{V}}^{\mathrm{I}}{\eta }_{1,3}+R_{\mathrm{V}}^{\mathrm{I}}S({\eta }_{4,6}){\epsilon }_{\mathrm{V}0}^{\mathrm{V}}+R_{\mathrm{V}}^{\mathrm{I}}S({\eta }_{4,6})R_0^{\mathrm{V}}{\epsilon }_{0,\mathrm{ee}}^0+J_{pos,man}{\eta }_{7,6+n}\end{array}$$ (10)

where ${\eta }_{i,j}$ is the matrix composed from the element ${\eta }_i$ to ${\eta }_j$, i.e., ${\eta }_{i,j}={[{\eta }_i,{\eta }_{i+1\dots }{\eta }_j]}^T$. $J_{pos,man}$ is the manipulator Jacobian matrix mapping $\dot{\mathrm{q}}$ to ${\dot{\epsilon }}_{0,\mathrm{ee}}^{\mathrm{I}}$.$\mathrm{S}(\cdot )$ is a function translating a vector $a={[ax,ay,az]}^T$ to a skew symmetric matrix as:

$$\mathrm{S}(a)=\left[\begin{array}{ccc}0& -az& ay\\ az& 0& -ax\\ -ay& ax& 0\end{array}\right]$$

Inspired by [29], ${\epsilon }_{0,\mathrm{ee}}^0$ can be further divided as:

$${\epsilon }_{0,\mathrm{ee}}^0{=[\mathrm{I}}_3,R_1^0\dots ,R_i^0\dots ,R_{n-1}^0]\cdot \mathrm{pp}\cdot a_{\mathrm{ee}}{=\mathrm{Y}}_{\mathrm{ee}}(q)a_{\mathrm{ee}}$$ (11)

where $\mathrm{pp}=\mathrm{diag}\left\{{\cos (\mathrm{q}}_1{),\sin (\mathrm{q}}_1),1,\dots {\cos (\mathrm{q}}_n{),\sin (\mathrm{q}}_n),1\right\}$ and $a_{\mathrm{ee}}={[a_1,a_1,d_1\dots a_n,a_n,d_n]}^T$. ${\mathrm{q}}_i$ is the position of the i-th joint, $a_{\mathrm{i}},d_{\mathrm{i}}$ are the link length and link offset associated with the $i$-th link. Besides,

$$J_{\mathrm{pos},\mathrm{man}}{\eta }_{7,6+n}=R_0^{\mathrm{I}}{\left[\mathrm{S}\right(\mathrm{z}}_0){\eta }_{7,6+n}{}_{(1)}{(\mathrm{D}}_n{-\mathrm{D}}_0)\mathrm{pp}\dots {\mathrm{S}(\mathrm{z}}_{n-1}){\eta }_{7,6+n}{}_{(n)}{(\mathrm{D}}_n{-\mathrm{D}}_{n-1}\left)\mathrm{pp}\right]a_{\mathrm{ee}}{=\mathrm{Y}}_{\mathrm{man}}(\eta ,q)a_{\mathrm{ee}}$$ (12)

where ${\mathrm{D}}_i=\mathrm{diag}\left\{{\mathrm{I}}_3^1\dots {\mathrm{I}}_3^i{,0}_{3\times 3}^{i+1}\dots 0_{3\times 3}^n\right\}$ and ${\mathrm{z}}_i$ is the unit vector of frame $i$ expressed in frame 0 of the manipulator.

On the other side,

$${\epsilon }_{\mathrm{I},\mathrm{ee}}^{\mathrm{I}}={\epsilon }_{\mathrm{I},\mathrm{V}}^{\mathrm{I}}+R_0^{\mathrm{I}}{\epsilon }_{\mathrm{V},0}^0+R_0^{\mathrm{I}}{\mathrm{Y}}_{\mathrm{ee}}(q)a_{\mathrm{ee}}={\mathrm{Y}}_{\mathrm{eeb}}(q)a_{\mathrm{eeb}}$$ (13)

where ${\mathrm{Y}}_{\mathrm{eeb}}(q)=[{\epsilon }_{\mathrm{I},\mathrm{V}}^{\mathrm{I}},R_0^{\mathrm{I}},R_0^{\mathrm{I}}{\mathrm{Y}}_{\mathrm{ee}}(q)]$, $a_{\mathrm{eeb}}={[I_{3\times 1},{\epsilon }_{\mathrm{V},0}^0,a_{\mathrm{ee}}^T]}^T$.

Therefore, by invoking (10)–(13), one can obtain:

$$Y_{\alpha }(\zeta ,\eta )=-[R_{\mathrm{V}}^{\mathrm{ee}}{\eta }_{1,3},R_{\mathrm{V}}^{\mathrm{ee}}S({\eta }_{4,6}){\epsilon }_{\mathrm{V}0}^{\mathrm{V}},R_{\mathrm{V}}^{\mathrm{ee}}S({\eta }_{4,6})R_0^{\mathrm{V}}{\mathrm{Y}}_{\mathrm{ee}}(q),R_{\mathrm{I}}^{\mathrm{ee}}{\mathrm{Y}}_{\mathrm{man}}(\eta ,q),\mathsf{\Upsilon }{\mathrm{Y}}_{\mathrm{eeb}}(q)]$$

$$a_{\alpha }={[\begin{array}{ccccc}{I}_{1\times 3}& I_{1\times 3}& a_{\mathrm{ee}}^T& a_{\mathrm{ee}}^T& a_{\mathrm{eeb}}^T\end{array}]}^T$$

and results can be further simplified as:

$$Y_{\alpha }(\zeta ,\eta )=-[R_{\mathrm{V}}^{\mathrm{ee}}{\eta }_{1,3}+R_{\mathrm{V}}^{\mathrm{ee}}S({\eta }_{4,6}){\mathrm{r}}_{\mathrm{V}0}^{\mathrm{V}},R_{\mathrm{V}}^{\mathrm{ee}}S({\eta }_{4,6})R_0^{\mathrm{I}}{\mathrm{Y}}_{\mathrm{ee}}(q)+R_{\mathrm{I}}^{\mathrm{ee}}{\mathrm{Y}}_{\mathrm{man}}(\eta ,q),\mathsf{\Upsilon }{\mathrm{Y}}_{\mathrm{eeb}}(q)]$$ (14)

$$a_{\alpha }=[\begin{array}{ccc}{I}_{1\times 3}& a_{\mathrm{ee}}^T& a_{\mathrm{eeb}}^T\end{array}{]}^T$$ (15)

To achieve the regressor matrix $Y_{\mathrm{zin}}(\zeta ,\eta )$, $Y_{\alpha }$ need to be transformed in to $Y_{\alpha s}\in {ℝ}^{1\times 3\mathrm{n}}$ as:

$$Y_{\alpha s}(\zeta ,\eta ){=[\mathrm{y}}_{\alpha 11},{\mathrm{y}}_{\alpha 12},{\mathrm{y}}_{\alpha 13},\dots {\mathrm{y}}_{\alpha n1},{\mathrm{y}}_{\alpha n2},{\mathrm{y}}_{\alpha n3}]$$

${\mathrm{y}}_{\alpha ij}$ is the (i,j) element of $Y_{\alpha }$.

Finally, from (5), (14), and (15), it can be obtained that:

$$Y_{\mathrm{zin}}(\zeta ,\eta )=\left[\begin{array}{cc}{Y}_{\alpha s}& 0\\ 0& Y_{\alpha s}\end{array}\right]$$ (16)

$$a_{z\mathrm{in}}={[{\overline{\mathrm{m}}}_1^T{a}_{\alpha 1}\dots {\overline{\mathrm{m}}}_1^T{a}_{\alpha \mathrm{n}},{\overline{\mathrm{m}}}_2^T{a}_{\alpha 1}\dots {\overline{\mathrm{m}}}_2^T{a}_{\alpha \mathrm{n}}]}^T$$ (17)

where $a_{\alpha \mathrm{i}}$ is the i-th element of the vector $a_{\alpha }$.□

Property 2.

For a vector $\varphi \in {ℝ}^{2\times 1}$ , $\dot{z}(\zeta )\varphi$ can be represented as the product of a regressor matrix and a constant vector.

$$\dot{z}(\zeta )\varphi =Y_{\mathrm{zd}}(\zeta ,\varphi )a_{\mathrm{zd}}$$ (18)

Proof. 

From (7) and the proof on the Property 1, it can be found that:

$$Y_{\mathrm{zd}}=\left[\begin{array}{c}{\varphi }_1{Y}_{\alpha s}(\zeta ,\dot{\zeta })\\ {\varphi }_2{Y}_{\alpha s}(\zeta ,\dot{\zeta })\end{array}\right]$$ (19)

$$a_{\mathrm{zd}}={[{\overline{\mathrm{m}}}_3^T{a}_{\alpha 1}\dots {\overline{\mathrm{m}}}_3^T{a}_{\alpha \mathrm{n}}]}^T$$ (20)

where ${\varphi }_i$ is the $i$-th element of the vector $\varphi$.□

Property 3.

For a vector $\varphi \in {ℝ}^{2\times 1}$ , $z(\zeta )\varphi$ can be represented as the product of a regressor matrix and a constant vector.

$$z(\zeta )\varphi =Y_{\mathrm{z}}(\zeta ,\varphi )a_z$$ (21)

Proof. 

According to (2), $z(\zeta )\varphi$ can be extended as:

$$\begin{array}{l}z(\zeta )\varphi ={\mathrm{m}}_3^T{\mathrm{T}}_{\mathrm{I}}^{\mathrm{ee}}\left[\begin{array}{c}{\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}\\ 1\end{array}\right]\varphi =({\displaystyle \sum _{i=1}^3{\mathrm{r}}_{i1}{\mathrm{m}}_{3i}^{}{\epsilon }_{\mathrm{I},fe1}^{\mathrm{I}}}+{\displaystyle \sum _{i=1}^3{\mathrm{r}}_{i2}{\mathrm{m}}_{3i}^{}{\epsilon }_{\mathrm{I},fe2}^{\mathrm{I}}}\\ +{\displaystyle \sum _{i=1}^3{\mathrm{r}}_{i3}{\mathrm{m}}_{3i}^{}{\epsilon }_{\mathrm{I},fe3}^{\mathrm{I}}}+{\overline{\mathrm{m}}}_3^T{R}_{\mathrm{I}}^{\mathrm{ee}}{\epsilon }_{\mathrm{ee},\mathrm{I}}^{\mathrm{I}}{+\mathrm{m}}_{34}^{})\left[\begin{array}{c}{\varphi }_1\\ {\varphi }_2\end{array}\right]\end{array}$$ (22)

where ${\mathrm{m}}_{3i}^{}$ and ${\epsilon }_{\mathrm{I},fei}^{\mathrm{I}}$ are the i-th elements of the vector ${\mathrm{m}}_3^{}$ and the vector ${\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}$, respectively. From (13), one obtains:

$$Y_z(\zeta ,\varphi )=\left[\begin{array}{ccc}\overline{R}{\varphi }_1& {\varphi }_1{\displaystyle \sum _{j=1}^3{\displaystyle \sum _{i=1}^3{r}_{ji}{y}_{\mathrm{eebj}}}}& {\varphi }_1\\ \overline{R}{\varphi }_2& {\varphi }_1{\displaystyle \sum _{j=1}^3{\displaystyle \sum _{i=1}^3{r}_{ji}{y}_{\mathrm{eebj}}}}& {\varphi }_2\end{array}\right]$$ (23)

$$a_z=[{\overline{\mathrm{m}}}_3^T{\epsilon }_{\mathrm{I},fe1}^{\mathrm{I}}{\overline{\mathrm{m}}}_3^T{\epsilon }_{\mathrm{I},fe2}^{\mathrm{I}}{\overline{\mathrm{m}}}_3^T{\epsilon }_{\mathrm{I},fe3}^{\mathrm{I}}-{(\mathrm{m}}_{31}{+\mathrm{m}}_{32}{+\mathrm{m}}_{33})a_{\mathrm{eeb}}^T{\mathrm{m}}_{34}{]}^T$$ (24)

where $\overline{R}{=[\mathrm{r}}_{11}{\mathrm{r}}_{21}{\mathrm{r}}_{31}{\mathrm{r}}_{12}{\mathrm{r}}_{22}{\mathrm{r}}_{32}{\mathrm{r}}_{13}{\mathrm{r}}_{23}{\mathrm{r}}_{33}]$, $y_{\mathrm{eeb}}{}_{\mathrm{j}}$ is the j-th row of the matrix ${\mathrm{Y}}_{\mathrm{eeb}}(q)$.□

Therefore, $J_{\mathrm{zin}}(\zeta )\eta$, $\dot{z}(\zeta )\varphi$, $z(\zeta )\varphi$ can be linearly separated as the product of the regressor matrices and the constant parameter vectors.

As described in previous proofs, camera perspective projection matrix $\mathrm{M}$, the manipulator kinematics parameters $a_{\mathrm{ee}}$ and the manipulator relative position vector ${\epsilon }_{\mathrm{V},0}^0$ are all collected in $a_{\mathrm{zin}}$,$a_{\mathrm{z}}$,$a_{\mathrm{zd}}$. The position of the target ${\epsilon }_{\mathrm{I},fe}^{\mathrm{I}}$ is collected in $a_{\mathrm{z}}$.

This work devotes to realize visual servoing under uncalibrated conditions. From the proofs of above three properties, uncertainties of the systems (i.e., uncertainties in intrinsic and extrinsic parameters of the camera, the kinematics of the UVMS, the position of the target w.r.t. the inertial frame) can be collected in these three parameter vectors in (17), (20), and (24). Because the target is fixed with the inertial frame, these vectors maintain constant and will be estimated by gradient update laws in the following section

By defining $\mathsf{\Delta }a=a-a_d$, a compensated depth-Jacobian matrix $J_{\mathsf{\Omega }}$ can be obtained by combining $J_{\mathrm{zin}}(\zeta )$ with $J_{\mathrm{zd}}(\zeta )$ [19,21].

$$J_{\mathsf{\Omega }}=J_{\mathrm{zin}}(\zeta )-\frac{x+x_d}{2}{J}_{\mathrm{zd}}(\zeta )=J(\zeta ,x)+\frac{\mathsf{\Delta }x}{2}{J}_{\mathrm{zd}}(\zeta )$$ (25)

where $x_d$ is the bounded desired position of the feature point on the image plane. Moreover, ${\dot{x}}_d$ and ${\ddot{x}}_d$ are the bounded desired velocity and desired acceleration of the feature point, respectively.

Substituting (3) and (4) into (25), (3) can be rewritten as:

$$J_{\mathsf{\Omega }}\dot{\zeta }=z\dot{x}+\frac{1}{2}\dot{z}\mathsf{\Delta }x\Rightarrow J_{\mathsf{\Omega }}\dot{\zeta }-z{\dot{x}}_d=z\mathsf{\Delta }\dot{x}+\frac{1}{2}\dot{z}\mathsf{\Delta }x$$ (26)

$J_{\mathsf{\Omega }}$ is used to describe the relationship from the generalized velocity term $\dot{\zeta }$ to $z\dot{x}+\frac{1}{2}\dot{z}\mathsf{\Delta }x$.

2.2. Dynamics

Coordinate frames of a typical UVMS with an eye-in-hand configuration camera can be represented in Figure 2.

Figure 2.

Coordinate frames of the UVMS.

Let ${\eta }_{\mathrm{ee}}=[{\epsilon }_{0,ee}^{\mathrm{I}}{}^T,{\phi }_{ee},{\theta }_{ee},{\psi }_{ee}{]}^T$ denote the position and Euler angle vector of the end-effector. The relationship between ${\dot{\eta }}_{\mathrm{ee}}$ and $\dot{\zeta }$ can be expressed as:

$${\dot{\eta }}_{\mathrm{ee}}=J_{\mathrm{uvms}}\dot{\zeta }$$ (27)

where $J_{\mathrm{uvms}}$ is the Jacobian matrix of the UVMS. Besides, the dynamic equation in the vehicle-joint space of UVMS can be written as [30]:

$$M(\zeta )\ddot{\zeta }+C(\zeta ,\dot{\zeta })\dot{\zeta }+D(\zeta ,\dot{\zeta })\dot{\zeta }+G(\zeta )={\tau }_{\mathrm{ctrl}}+{\tau }_{\mathrm{ext}}$$ (28)

$M(\zeta )\in {ℝ}^{(6+\mathrm{n})\times (6+\mathrm{n})}$ denotes the mass matrix including the added mass term, $C(\zeta ,\dot{\zeta })\in {ℝ}^{(6+\mathrm{n})\times (6+\mathrm{n})}$ represents the centrifugal and Coriolis effect matrix, $D(\zeta ,\dot{\zeta })\in {ℝ}^{(6+\mathrm{n})\times (6+\mathrm{n})}$ is the contribution due to the damping effects, $G(\zeta )\in {ℝ}^{(6+\mathrm{n})\times 1}$ represents the restoring matrix. ${\tau }_{\mathrm{ctrl}}\in {ℝ}^{(6+\mathrm{n})\times 1}$ is the controller output, ${\tau }_{\mathrm{ext}}\in {ℝ}^{(6+\mathrm{n})\times 1}$ represents the bounded unknown external disturbances on the UVMS, mainly caused by underwater currents, variation in load.

Specifically speaking, one can extend these dynamics matrices as:

$$M(\zeta )=\left[\begin{array}{cc}{M}_v({\eta }_v)& M_{vm}(q)\\ M_{vm}^T(q)& M_{\mathrm{m}}(q)\end{array}\right]$$

$$C(\zeta ,\dot{\zeta })=\left[\begin{array}{cc}{C}_v({\eta }_v,{\dot{\eta }}_v)& 0\\ 0& C_{\mathrm{m}}(q,\dot{q})\end{array}\right]$$

$$D(\zeta ,\dot{\zeta })=\left[\begin{array}{cc}{D}_v({\eta }_v,{\dot{\eta }}_v)& 0\\ 0& D_{\mathrm{m}}(q,\dot{q})\end{array}\right]$$

$$G(\zeta )=\left[\begin{array}{l}{G}_v({\eta }_v)\\ G_{\mathrm{m}}(q)\end{array}\right]$$

Subscript $v$ and $\mathrm{m}$ represent vehicle and manipulator, respectively. $M_v\in {ℝ}^{6\times 6}$ is the mass matrix of the sole vehicle, $M_{\mathrm{m}}\in {ℝ}^{\mathrm{nxn}}$ is the mass matrix of the manipulator. $M_{\mathrm{vm}}\in {ℝ}^{6\mathrm{xn}}$ is the coupling term between the vehicle and the manipulator. Similar properties can also be found in $C(\zeta ,\dot{\zeta })$, $D(\zeta ,\dot{\zeta })$ and $G(\zeta )$. Comparing with traditional underwater vehicles and fixed manipulators, UVMSs are more complex due to multibody inner coupling effects.

Inspired by the inherent dynamic characteristics of UVMSs, $\widehat{M}$ and $\widehat{C}$ are chosen to satisfy following properties [30,31]:

Property 4.

The inertial matrix $M(\zeta )$ is, positive and bounded:

$$\begin{gathered} M(\zeta )>0 \\ {\upsilon }_1\mathrm{I}\le M(\zeta )\le {\upsilon }_2\mathrm{I} \end{gathered}$$

${\upsilon }_1$ and ${\upsilon }_2$ are two positive constants.

Property 5.

The 2-norm of Coriolis matrix $C(\zeta ,\dot{\zeta })$ is bounded:

$$\Vert C(\zeta ,\dot{\zeta })\Vert \le {\upsilon }_3{\Vert \dot{\zeta }\Vert }_{\max }$$

where ${\upsilon }_3$ is a positive scalar. The 2-norm means:

$$x\in R^{\mathrm{n}}\Rightarrow \Vert x\Vert =\sqrt{x^{T}x},x\in R^{\mathrm{nxn}}\Rightarrow \Vert x\Vert =\sqrt{{\lambda }_{\max }(x^{T}x})$$

where ${\lambda }_{\max }(\cdot )$ is the maximum eigenvalue of a matrix.

Property 6.

The matrix $\dot{M}(\zeta )-2C(\zeta ,\dot{\zeta })$ is skew-symmetric:

$$\dot{M}(\zeta )-2C(\zeta ,\dot{\zeta })=-{\left[\dot{M}(\zeta )-2C(\zeta ,\dot{\zeta })\right]}^T$$

From Properties 5 and 6, it can be found that:

$$\Vert \dot{M}(\zeta )\Vert \le 2{\upsilon }_3{\Vert \dot{\zeta }\Vert }_{\max }$$

which means the 2-norm of $\dot{M}(\zeta )$ is also upper bounded.

Intrinsic dynamic parameters of a UVMS, e.g., the mass and the inertia tensor of each DOF, the buoyance of each DOF etc., are always difficult to accurately obtain. Hence, it is also difficult to acquire the accurate dynamic modeling in practice. Unmodeled dynamics, in addition to external disturbances are considered as system lumped disturbances. Then, the dynamic Equation (27) can be furthered represented as:

$$\widehat{M}(\zeta )\ddot{\zeta }+\widehat{C}(\zeta ,\dot{\zeta })\dot{\zeta }+\widehat{D}(\zeta ,\dot{\zeta })\dot{\zeta }+\widehat{G}(\zeta )={\tau }_{\mathrm{ctrl}}+{\tau }_{\mathrm{d}}$$ (29)

where $\widehat{M}(\zeta )$, $\widehat{C}(\zeta ,\dot{\zeta })$, $\widehat{D}(\zeta ,\dot{\zeta })$, $\widehat{G}(\zeta )$ are nominal dynamic model matrices. $\tilde{M}(\zeta )$, $\tilde{C}(\zeta ,\dot{\zeta })$, $\tilde{D}(\zeta ,\dot{\zeta })$, $\tilde{G}(\zeta )$ are modeling error terms, and they are defined as: $\tilde{A}=A-\widehat{A}$. ${\tau }_{\mathrm{d}}={\tau }_{\mathrm{ext}}-(\tilde{M}\ddot{\zeta }+\tilde{C}\dot{\zeta }+\tilde{D}\dot{\zeta }+\tilde{G})$ denotes the time varying disturbance term.

Restoring moments are caused by gravity and buoyancy forces. For the UVMS, the vehicle is easily affected by the restoring moments because of the movement of the manipulator. Therefore, it is important to decrease the restoring moments by choosing proper $\dot{\zeta }$. We assume that $\Vert \widehat{G}(\zeta )\Vert$, $\Vert \frac{\partial \widehat{G}(\zeta )}{\partial \zeta }\Vert$ and $\Vert \frac{{\partial }^2\widehat{G}(\zeta )}{\partial {\zeta }^2}\Vert$ are all bounded.

Remark 1.

Nominal dynamic model matrices $\widehat{M}(\zeta )$ , $\widehat{C}(\zeta ,\dot{\zeta })$ are chosen to satisfy Properties 4–6. In the following controller, $\widehat{M}(\zeta )$ , $\widehat{C}(\zeta ,\dot{\zeta })$ , $\widehat{D}(\zeta ,\dot{\zeta })$ and $\widehat{G}(\zeta )$ are obtained through choosing another set of intrinsic dynamic parameters (which is different from the actual intrinsic parameters, and it is utilized in forward dynamic routines in simulations), rather than updated by adaptive laws. In other word, we do not estimate or update nominal matrices, we directly calculate them by utilizing system states $[\zeta ,\dot{\zeta }]$ and intrinsic dynamic parameters. Therefore, $\widehat{M}(\zeta )$ , $\widehat{C}(\zeta ,\dot{\zeta })$ , $\widehat{D}(\zeta ,\dot{\zeta })$ and $\widehat{G}(\zeta )$ can be chosen to satisfy above properties.

3. Controller Design

The control objective is to realize the convergence of image errors. In the process of designing the controller, at first, we establish a new reference velocity term to optimize the restoring moment through using the gradient projection method (GPM), moreover, novel adaptive laws are established to estimate the uncertainties in uncalibrated systems by utilizing the linear separation method which we described in the last section. Secondly, we employ a high-order disturbance observer to compensate the time-varying lumped disturbance term in (29). Thirdly, we propose a novel composite controller to guarantee the convergence of image errors, stability analysis based on the Lyapunov theory is also presented.

3.1. Adaptive Laws

To realize visual servoing, we should firstly design a reference velocity term in vehicle-joint space. If the kinematic parameters are available, then $a_{\mathrm{zin}}$,$a_{\mathrm{z}}$,$a_{\mathrm{zd}}$ can be obtained. A reference velocity can be designed as:

$${\dot{\zeta }}_r=J^{†}{}_{\mathsf{\Omega }}z{\dot{x}}_d-{\alpha }_v{J}^T{}_{\mathsf{\Omega }}z\mathsf{\Delta }x$$ (30)

where ${\alpha }_v$ is a positive gain. Then, if there exists an ideal controller to make $\dot{\zeta }={\dot{\zeta }}_r$, substituting (30) into (26), one can find:

$$-{\alpha }_v{J}_{\mathsf{\Omega }}{J}^T{}_{\mathsf{\Omega }}z\mathsf{\Delta }x=z\mathsf{\Delta }\dot{x}+\frac{1}{2}\dot{z}\mathsf{\Delta }x$$ (31)

Consider a Lyapunov function as:

$$V_{\mathrm{temp}}=\frac{1}{2}z\mathsf{\Delta }{x}^T\mathsf{\Delta }x,$$

take the time-derivative of $V_{\mathrm{temp}}$, yields:

$${\dot{V}}_{\mathrm{temp}}=-{\alpha }_v\mathsf{\Delta }{x}^T{J}_{\mathsf{\Omega }}{J}^T{}_{\mathsf{\Omega }}z\mathsf{\Delta }x.$$

If $J_{\mathsf{\Omega }}$ is non-singular, ${\dot{V}}_{\mathrm{temp}}<0$ and $\mathsf{\Delta }x\to 0$ as $t\to \infty$.

To achieve tracking control, it is expected to acquire the accurate value of $J_{\mathsf{\Omega }}$, $z$ and $\dot{z}$, which equals to acquire the accurate value of $a_{\mathrm{zin}}$, $a_{\mathrm{z}}$ and $a_{\mathrm{zd}}$. Unfortunately, it cannot be accurately obtained by considering the kinematic uncertainties of the system. Hence it is necessary to build adaptive laws to estimate them. The core is to estimate $a_{\mathrm{zin}}$, $a_{\mathrm{z}}$ and $a_{\mathrm{zd}}$.

The compensated depth-Jacobian matrix in (25) can be estimated as:

$${\widehat{J}}_{\mathsf{\Omega }}=\widehat{J}(\zeta ,x)+\frac{\mathsf{\Delta }x}{2}{\widehat{J}}_{\mathrm{zd}}(\zeta )$$ (32)

By employing the pseudo inverse and transpose of ${\widehat{J}}_{\mathsf{\Omega }}$ in (32) and the estimation of depth $\widehat{z}$, a reference velocity term that similar with the ones in [19,20,21] can be represented as:

$${\dot{\zeta }}_r={\widehat{J}}^{†}{}_{\mathsf{\Omega }}\widehat{z}{\dot{x}}_d-{\alpha }_v{\widehat{J}}^T{}_{\mathsf{\Omega }}\widehat{z}\mathsf{\Delta }x$$ (33)

where ${\widehat{J}}^{†}{}_{\mathsf{\Omega }}$ denotes the pseudo inverse of ${\widehat{J}}_{\mathsf{\Omega }}$, $\widehat{z}$ is the estimated depth.

The UVMS possesses relatively high DOFs compared with simple manipulators in [19,20,21], hence the redundancy of the UVMS should be taken into consideration when design the reference velocity. Because the UVMS is easily be affected by the restoring moment, the GPM is utilized to optimize the restoring moments through exploiting kinematic redundancy, a novel reference velocity can be represented as:

$${\dot{\zeta }}_r={\widehat{J}}^{†}{}_{\mathsf{\Omega }}\widehat{z}{\dot{x}}_d-{\alpha }_v{\widehat{J}}^T{}_{\mathsf{\Omega }}\widehat{z}\mathsf{\Delta }x-{\kappa }_v(I-{\widehat{J}}^{†}{}_{\mathsf{\Omega }}{\widehat{J}}^{}{}_{\mathsf{\Omega }})\nabla P_r$$ (34)

where ${\kappa }_v$ is a positive gain. $\nabla P_r$ is defined as $\nabla P_r={\widehat{G}}^T(\zeta )W_G^{}\frac{\partial \widehat{G}(\zeta )}{\partial \zeta }$, $W_G^{}$ is a positive weight matrix. $(I-{\widehat{J}}^{†}{}_{\mathsf{\Omega }}{\widehat{J}}^{}{}_{\mathsf{\Omega }})$ projects $\nabla P_r$ into the null space of ${\widehat{J}}^{}{}_{\mathsf{\Omega }}$, hence $\nabla P_r$ is only effective in the null space.

Although some previous works utilize the GPM to optimize restoring moments for UVMSs [32,33,34], there are two main differences. One is the Jacobian matrix in [32,33,34] is used to describe the velocity relationship between the configuration space and the Cartesian space (i.e., the Jacobian matrix in (27)), rather than the Jacobian matrix described in (26). The other one is previous works [32,33,34] all depend on the accurate Jacobian matrix, rather than the estimated Jacobian matrix.

Then, a sliding vector can be defined as:

$$s=\dot{\zeta }-{\dot{\zeta }}_r$$ (35)

estimation error terms can be defined as: $\mathsf{\Delta }{a}_{\mathrm{zin}}^{}={\widehat{a}}_{\mathrm{zin}}^{}-a_{\mathrm{zin}}^{}$, $\mathsf{\Delta }{a}_{\mathrm{zd}}^{}={\widehat{a}}_{\mathrm{zd}}^{}-a_{\mathrm{zd}}^{}$, $\mathsf{\Delta }{a}_{\mathrm{z}}^{}={\widehat{a}}_{\mathrm{z}}^{}-a_{\mathrm{z}}^{}$. Then, substituting (7), (18) into (25), one can find:

$$({\widehat{J}}_{\mathsf{\Omega }}-J_{\mathsf{\Omega }})\dot{\zeta }=({\widehat{J}}_{\mathrm{zin}}(\zeta )-J_{\mathrm{zin}}(\zeta ))\dot{\zeta }-\frac{x+x_d}{2}({\widehat{J}}_{\mathrm{zd}}(\zeta )-J_{\mathrm{zd}}(\zeta ))\dot{\zeta }=Y_{\mathrm{zin}}(\zeta ,\dot{\zeta })\mathsf{\Delta }{a}_{\mathrm{zin}}-\frac{1}{2}{Y}_{\mathrm{zd}}(\zeta ,\dot{\zeta },x+x_d)\mathsf{\Delta }{a}_{\mathrm{zd}}$$ (36)

From (21) and (34), one has:

$${\widehat{J}}_{\mathsf{\Omega }}{\dot{\zeta }}_r=\widehat{z}{\dot{x}}_d-{\alpha }_v\widehat{z}{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x=Y_z(\zeta ,{\dot{x}}_d-{\alpha }_v{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x)\mathsf{\Delta }{a}_z$$ (37)

Invoking (26), (34)–(36), the closed-loop kinematics system can be represented as:

$$z\mathsf{\Delta }\dot{x}+\frac{1}{2}\dot{z}\mathsf{\Delta }x=Y_z(\zeta ,{\dot{x}}_d-{\alpha }_v{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x)\mathsf{\Delta }{a}_z+{\widehat{J}}_{\mathsf{\Omega }}s+Y_{zd}(\zeta ,\dot{\zeta },\frac{x+x_d}{2})\mathsf{\Delta }{a}_{zd}-Y_{zin}(\zeta ,\dot{\zeta })\mathsf{\Delta }{a}_{zin}-{\alpha }_{v}z{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x$$ (38)

To calculate the reference velocity term in (34) and guarantee the stability of the system, it is necessary to estimate the unknown parameters of the systems (i.e., the constant vectors in (8), (18) and (21)). The adaptive update laws are designed as:

$${\dot{\widehat{a}}}_{\mathrm{zin}}={\lambda }_{\mathrm{zin}}{Y}_{\mathrm{zin}}^{\mathrm{T}}(\zeta ,\dot{\zeta })\mathsf{\Delta }x$$ (39)

$${\dot{\widehat{a}}}_{\mathrm{zd}}=-{\lambda }_{\mathrm{zd}}{Y}_{\mathrm{zd}}^{\mathrm{T}}(\zeta ,\dot{\zeta },\frac{x+x_d}{2})\mathsf{\Delta }x$$ (40)

$${\dot{\widehat{a}}}_{\mathrm{z}}=-{\lambda }_{\mathrm{z}}{Y}_{\mathrm{z}}^{\mathrm{T}}(\zeta ,{\dot{x}}_d-\alpha {\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}^T{}_{\mathsf{\Omega }}\mathsf{\Delta }x)\mathsf{\Delta }x$$ (41)

${\lambda }_{\mathrm{zin}}$, ${\lambda }_{\mathrm{zd}}$ and ${\lambda }_{\mathrm{z}}$ are the positive gain matrices, respectively.

The novel reference velocity term (34), update laws (39)–(41) and stability analysis in the end of this section is one of the main contributions of this work.

Remark 2.

We assume that the update laws (39) and (40) will not lead to the singularity of ${\widehat{J}}_{\mathsf{\Omega }}$ . In other words, ${\widehat{J}}_{\mathsf{\Omega }}$ is supposed to be non-singular. This is a common assumption in uncalibrated visual servoing works [19,20,21]. Besides, one may adopt the strategy presented in [35] to avoid the singular on ${\widehat{J}}_{\mathsf{\Omega }}$ .

3.2. High Order Disturbance Observer

When compared with nonlinear disturbance observer in [31,36], the performance under high order disturbance observer (HODO) is better, because the disturbance model refers to high order information of the disturbance. Besides, in order to prove the asymptotic stability of the system, the disturbance term is often supposed to be constant in previous works [31,36]. This assumption can be cancelled under the high order disturbance observer.

The purpose of employing a disturbance observer is to estimate and compensate the time-varying unknown lumped disturbance term ${\tau }_{\mathrm{d}}$. The philosophy of designing the disturbance observer is to estimate the unmeasurable term ${\tau }_{\mathrm{d}}$ through measurable states. To establish the high order disturbance observer, we first rewrite the closed loop system (29) as a form as state space:

$$\{\begin{array}{l}{\dot{x}}_1=x_2\\ {\dot{x}}_2=\underset{f}{\underbrace{{\widehat{M}}^{-1}({\tau }_{\mathrm{ctrl}}-\widehat{C}{x}_2-\widehat{D}{x}_2-\widehat{G})}}+\underset{d}{\underbrace{{\widehat{M}}^{-1}{\tau }_{\mathrm{d}}}}\end{array}$$ (42)

where $x_1=\zeta$, $x_2=\dot{\zeta }$ denote the system variable states. Besides, $f={\widehat{M}}^{-1}({\tau }_{\mathrm{ctrl}}-\widehat{C}{x}_2-\widehat{D}{x}_2-\widehat{G})$ is available in the system, $d={\widehat{M}}^{-1}{\tau }_{\mathrm{d}}$ is the unknown disturbance term to be estimated.

Suppose the disturbance term $d$ has a bounded $r^{th}$ derivative,

$$\left|d^{(r)}\right|\le {\delta }_0$$

where ${\delta }_0$ is the upper bound. The disturbance term $d$ can be described through a linear system, given by [37]

$$\{\begin{array}{l}\dot{\xi }=W\xi +E{d}^{(r)}\\ d=L\xi \end{array}$$ (43)

where $\xi \in {ℝ}^{(6+n)·r}$ is an auxiliary variable. $W\in {ℝ}^{(6+n)·r\times (6+n)·r}$, $E\in {ℝ}^{(6+n)·r\times (6+n)}$, and $L\in {ℝ}^{(6+n)\times (6+n)·r}$ can be represented as the following forms:

$$W=\left[\begin{array}{cc}{0}_{(6+n)·(r-1)\times (6+n)}& I_{(6+n)·(r-1)}\\ 0_{(6+n)\times (6+n)}& 0_{(6+n)\times (6+n)·(r-1)}\end{array}\right]$$

$$E=\left[\begin{array}{c}{0}_{(6+n)·(r-1)\times (6+n)}\\ I_{6+n}\end{array}\right]$$

$$L=[\begin{array}{cc}{I}_{6+n}& 0_{(6+n)\times (6+n)·(r-1)}\end{array}]$$

Combining (42) and (43), one can obtains:

$$\{\begin{array}{l}\dot{\xi }=W\xi +E{D}^{(r)}\\ {\dot{x}}_2=f+L\xi \end{array}$$ (44)

Then a new extended system can be derived through introducing an auxiliary variable $\gamma ={\left[\begin{array}{cc}{\gamma }_1& {\gamma }_2\end{array}\right]}^T$ as follows:

$$\left[\begin{array}{c}{\gamma }_1\\ {\gamma }_2\end{array}\right]=\left[\begin{array}{c}{x}_2\\ -p(x_2)+\xi \end{array}\right]$$ (45)

where $p(x_2)$ is a polynomial vector to be designed. Differentiating (45) with respect to time and substituting (44), the dynamics of $\gamma$ can be represented as:

$$\{\begin{array}{l}{\dot{\gamma }}_1=f+L(p({\gamma }_1)+{\gamma }_2)\\ {\dot{\gamma }}_2=(W-l({\gamma }_1)L){\gamma }_2+T+E{D}^{(r)}\end{array}$$ (46)

where $l({\gamma }_1)=\frac{\partial p({\gamma }_1)}{\partial {\gamma }_1}$ is a vector to be designed. $T=Wp({\gamma }_1)-l({\gamma }_1)(f+Lp({\gamma }_1))$, it is available through system measurable states and pre-designed vector $l({\gamma }_1)$.

Then, a high order disturbance observer can be derived as:

$$\{\begin{array}{l}{\dot{\widehat{\gamma }}}_2=(W-l({\gamma }_1)L){\widehat{\gamma }}_2+T\\ \widehat{\xi }={\widehat{\gamma }}_2+p({\gamma }_1)\\ \widehat{d}=L\widehat{\xi }\end{array}$$ (47)

Finally, the estimation of ${\tau }_{\mathrm{d}}$ follows:

$${\widehat{\tau }}_{\mathrm{d}}=\widehat{M}\widehat{d}$$ (48)

The estimation error is defined as $e_{\xi }=\xi -\widehat{\xi }$. And it is governed by following dynamic system:

$${\dot{e}}_{\xi }=\dot{\xi }-\dot{\widehat{\xi }}=W\xi +E{d}^{(r)}-(W-l({\gamma }_1)L){\widehat{\gamma }}_2-T-\dot{p}({\gamma }_1)=(W-l({\gamma }_1)L)e_{\xi }+E{d}^{(r)}$$ (49)

The principle of designing $l({\gamma }_1)$ is to make the dominant pole of (49) lie in the left-half plane, then the system described in (49) is bounded-input bounded-output (BIBO) stable. If $l({\gamma }_1)$ is available, then $p({\gamma }_1)$ can be obtained from the relationship $l({\gamma }_1)=\frac{\partial p({\gamma }_1)}{\partial {\gamma }_1}$. Since the system is BIBO stable, the disturbance estimation error is bounded and satisfies:

$$\Vert d-\widehat{d}\Vert \le {\delta }_1$$

where ${\delta }_1$ is the upper bound. Since $\widehat{M}$ is bounded, it can be deduced that:

$$\Vert {\tau }_{\mathrm{d}}-{\widehat{\tau }}_{\mathrm{d}}\Vert =\Vert \widehat{M}(d-\widehat{d})\Vert \le \Vert \widehat{M}\Vert \Vert (d-\widehat{d})\Vert \le \sigma$$

where $\sigma$ is an unknown bounded constant.

3.3. Composite Controller

Now, by using the reference velocity ${\dot{\zeta }}_r$, the sliding vector $s$, the image error $\mathsf{\Delta }x$, the disturbance estimation ${\widehat{\tau }}_{\mathrm{d}}$ and the approximation of unknown bound $\widehat{\sigma }$, a composite controller with the HODO for uncalibrated visual servoing can be designed as:

$${\tau }_{\mathrm{ctrl}}=\widehat{M}({\ddot{\zeta }}_{\mathrm{r}}-K_{p}s)-K_s{\widehat{J}}^T{}_{\mathsf{\Omega }}\mathsf{\Delta }x+\widehat{C}{\dot{\zeta }}_r+\widehat{D}\dot{\zeta }+\widehat{G}-{\widehat{\tau }}_{\mathrm{d}}-\widehat{\sigma }\mathrm{sgn}\left(\mathrm{s}\right)$$ (50)

where $K_p$ and $K_s$ are positive diagonal matrices. In the controller, the first two terms are the feedback control term in vehicle-joint space and image space, receptively. $\widehat{C}{\dot{\zeta }}_r+\widehat{D}\dot{\zeta }+\widehat{G}$ is the feed-forward control term. In vehicle-joint space, the reference velocity term ${\dot{\zeta }}_r$ is provided by (34), (39)–(41) through estimating $J_{\mathsf{\Omega }}$ and $z$. ${\widehat{\tau }}_{\mathrm{d}}$ is the estimation term from the disturbance observer in (48). $\widehat{\sigma }$ is the estimated value of the constant bound $\sigma$, and it is updated as the following adaptive law:

$$\dot{\widehat{\sigma }}={\lambda }_{\sigma }\Vert s\Vert$$ (51)

Substituting (50) into (29), The close-loop dynamics equation of the UVMS can be represented as:

$$\widehat{M}\dot{s}=-\widehat{M}{K}_{p}s-\widehat{C}s+\widehat{M}\tilde{d}-K_s{\widehat{J}}^T{}_{\mathsf{\Omega }}\mathsf{\Delta }x-\widehat{\sigma }\mathrm{sgn}\left(\mathrm{s}\right)$$ (52)

The whole architecture of the proposed scheme can be found in Figure 3.

Figure 3.

The architecture of the proposed scheme.

Theorem 1.

The adaptive laws (39)–(41), the high order disturbance observer (48), and the composite controller (50) guarantee the convergence of the image space tracking errors, i.e., $\mathsf{\Delta }x\to 0$ , $\mathsf{\Delta }\dot{x}\to 0$ as $t\to \infty$ .

Proof. 

Consider a Lyapunov candidate function as:

$$\begin{array}{l}V=\frac{1}{2}(z\mathsf{\Delta }{x}^T\mathsf{\Delta }x+\mathsf{\Delta }{a}_{\mathrm{zin}}^T{\lambda }_{\mathrm{zin}}^{-1}\mathsf{\Delta }{a}_{\mathrm{zin}}^{}+\mathsf{\Delta }{a}_{\mathrm{zd}}^T{\lambda }_{\mathrm{zd}}^{-1}\mathsf{\Delta }{a}_{\mathrm{zd}}^{}+\mathsf{\Delta }{a}_{\mathrm{z}}^T{\lambda }_{\mathrm{z}}^{-1}\mathsf{\Delta }{a}_{\mathrm{z}}^{}\\ +s^T{K}_s^{-1}\widehat{M}s+\frac{\Vert K_s^{-1}\Vert }{{\lambda }_{\sigma }}{\tilde{\sigma }}^2)\end{array}$$ (53)

From Properties 1–3, unknown parameter vectors expressed in (8), (18) and (21) are all constant. By differentiating (53) w.r.t. time, one has

$$\begin{array}{l}\dot{V}=\frac{1}{2}\mathsf{\Delta }{x}^T\dot{z}\mathsf{\Delta }x+\mathsf{\Delta }{x}^{T}z\mathsf{\Delta }\dot{x}+\mathsf{\Delta }{a}_{\mathrm{zin}}^T{\lambda }_{\mathrm{zin}}^{-1}{\dot{\widehat{a}}}_{\mathrm{zin}}^{}+\mathsf{\Delta }{a}_{\mathrm{zd}}^T{\lambda }_{\mathrm{zd}}^{-1}{\dot{\widehat{a}}}_{\mathrm{zd}}^{}+\mathsf{\Delta }{a}_{\mathrm{z}}^T{\lambda }_{\mathrm{z}}^{-1}{\dot{\widehat{a}}}_{\mathrm{z}}^{}\\ +s^T{K}_s^{-1}\widehat{M}\dot{s}+\frac{1}{2}{s}^T{K}_s^{-1}\dot{\widehat{M}}s-\frac{\Vert K_s^{-1}\Vert }{{\lambda }_{\sigma }}\dot{\widehat{\sigma }}\tilde{\sigma }\end{array}$$ (54)

take Property 6, (38)–(41), (50) into (54), one has:

$$\begin{array}{l}\dot{V}=-{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x+\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}^{}s-s^T{K}_s^{-1}\widehat{M}{K}_{p}s+s^T{K}_s^{-1}\widehat{M}\tilde{d}-s^T{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x\\ -\widehat{\sigma }{s}^T{K}_s^{-1}\mathrm{sgn}\left(\mathrm{s}\right)-\frac{\Vert K_s^{-1}\Vert }{{\lambda }_{\sigma }}\dot{\widehat{\sigma }}\tilde{\sigma }\\ =-{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x-s^T\widehat{M}{K}_{p}s+s^T{K}_s^{-1}\widehat{M}\tilde{d}-\widehat{\sigma }{s}^T{K}_s^{-1}\mathrm{sgn}\left(\mathrm{s}\right)-\tilde{\sigma }\Vert K_s^{-1}\Vert \Vert s\Vert \end{array}$$ (55)

Let $s^T{K}_s^{-1}={\eta }_{\mathrm{s}}$, since $K_s$ is a positive diagonal matrix, it can be found:

$${\mathrm{sgn}(\mathrm{s}}^T)=\mathrm{sgn}({\eta }_{\mathrm{s}})$$

since ${\widehat{J}}_{\mathsf{\Omega }}$ is non-singular and $\widehat{M}$ is positive, one can find a nonnegative definite continuous function $W(\zeta )$ satisfies:

$$\begin{array}{l}\dot{V}=-{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x-s^T{K}_s^{-1}\widehat{M}{K}_{p}s+{\eta }_{\mathrm{s}}\widehat{M}\tilde{d}-\widehat{\sigma }{\eta }_{\mathrm{s}}\mathrm{sgn}({\eta }_{\mathrm{s}}^T)-\tilde{\sigma }\Vert K_s^{-1}\Vert \Vert s\Vert \\ \le -{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x-s^T\widehat{M}{K}_{p}s+\sigma \Vert {\eta }_{\mathrm{s}}\Vert -\widehat{\sigma }\Vert {\eta }_{\mathrm{s}}\Vert -\tilde{\sigma }\Vert {\eta }_{\mathrm{s}}\Vert \\ =-{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x-s^T\widehat{M}{K}_{p}s\\ =-W(\zeta )\le 0\end{array}$$ (56)

Integrate $W(\zeta )$ from time 0 to current time $T$, it can be obtained

$${\displaystyle {\int }_0^{T}W(\zeta (r))}dr\le V(\zeta (0))-V(\zeta (T))\le V(\zeta (0))$$ (57)

$V(\zeta (0))$ is fixed by the choice of initial condition, therefore ${\displaystyle {\int }_0^{T}W(\zeta (r))}dr$ is bounded.

Since ${\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T$ is positive definite, from (56), $\dot{V}$ is negative semi-definite, which implies that $\mathsf{\Delta }x$,$\mathsf{\Delta }{a}_{\mathrm{zin}}^{}$,$\mathsf{\Delta }{a}_{\mathrm{zd}}^{}$, $\mathsf{\Delta }{a}_{\mathrm{z}}^{}$,$s$ and $\tilde{\sigma }$ are all bounded. Therefore, ${\widehat{a}}_{\mathrm{zin}}$, ${\widehat{a}}_{\mathrm{zd}}$, ${\widehat{a}}_{\mathrm{z}}$ and $\widehat{\sigma }$ are all bounded. Then the boundness of ${\widehat{J}}_{\mathrm{zin}}(\zeta )$, $\dot{\widehat{z}}$, ${\widehat{J}}_z(\zeta )$ and $\widehat{z}$ can be obtained. Hence, the boundness ${\widehat{J}}_{\mathsf{\Omega }}$ can be concluded from (32). Since ${\dot{x}}_d$ and $x_d$ are all bounded, ${\dot{\zeta }}_r$ is also bounded. Because $s$ and ${\dot{\zeta }}_r$ are all bounded, the boundness of $\dot{\zeta }$ can be achieved. From (3) and (7), it can be obtained that $\dot{x}$ and $\dot{z}$ are all bounded, hence the boundness of $\mathsf{\Delta }\dot{x}$ can be achieved. From (39)–(41), the boundness of ${\dot{\widehat{a}}}_{\mathrm{zin}}$,${\dot{\widehat{a}}}_{\mathrm{zd}}$ and ${\dot{\widehat{a}}}_{\mathrm{z}}$ can be obtained. Therefore, the boundness of ${\dot{\widehat{J}}}_z(\zeta )$ and ${\dot{\widehat{J}}}_{\mathrm{zin}}(\zeta )$ can be achieved by differentiating (7) and (8), respectively. As ${\dot{\widehat{J}}}_{\mathrm{zin}}(\zeta )-\frac{\dot{x}+{\dot{x}}_d}{2}{\widehat{J}}_z(\zeta )-\frac{x+x_d}{2}{\dot{\widehat{J}}}_z(\zeta )={\dot{\widehat{J}}}_{\mathsf{\Omega }}$, the boundness of ${\dot{\widehat{J}}}_{\mathsf{\Omega }}$ can also be achieved.

Differentiating (34) w.r.t. time, one has:

$$\begin{array}{l}{\ddot{\zeta }}_r={\dot{\widehat{J}}}^{†}{}_{\mathsf{\Omega }}\widehat{z}{\dot{x}}_d+{\widehat{J}}^{†}{}_{\mathsf{\Omega }}\dot{\widehat{z}}{\dot{x}}_d+{\widehat{J}}^{†}{}_{\mathsf{\Omega }}\widehat{z}{\ddot{x}}_d-\alpha {\dot{\widehat{J}}}^T{}_{\mathsf{\Omega }}\widehat{z}\mathsf{\Delta }x-{\alpha }_v{\widehat{J}}^T{}_{\mathsf{\Omega }}\dot{\widehat{z}}\mathsf{\Delta }x-{\alpha }_v{\widehat{J}}^T{}_{\mathsf{\Omega }}\widehat{z}\mathsf{\Delta }\dot{x}\\ +{\kappa }_v({\dot{\widehat{J}}}^{†}{}_{\mathsf{\Omega }}{\widehat{J}}^{}{}_{\mathsf{\Omega }}+{\widehat{J}}^{†}{}_{\mathsf{\Omega }}{\dot{\widehat{J}}}^{}{}_{\mathsf{\Omega }})\nabla P_r-{\kappa }_v(I-{\widehat{J}}^{†}{}_{\mathsf{\Omega }}{\widehat{J}}^{}{}_{\mathsf{\Omega }}){\nabla }^2{P}_r\dot{\zeta }\end{array}$$ (58)

because ${\dot{\widehat{J}}}_{\mathsf{\Omega }}$, $\dot{\widehat{z}}$, ${\dot{x}}_d$, ${\ddot{x}}_d$, $\mathsf{\Delta }\dot{x}$, $\dot{\zeta }$,$\Vert \widehat{G}(\zeta )\Vert$, $\Vert \frac{\partial \widehat{G}(\zeta )}{\partial \zeta }\Vert$ and $\Vert \frac{{\partial }^2\widehat{G}(\zeta )}{\partial {\zeta }^2}\Vert$ are all bounded, the boundness of ${\ddot{\zeta }}_r$ and $\dot{s}$ can be achieved. Differentiating $W$ w.r.t. time, it satisfies:

$$\begin{array}{l}\dot{W}={\alpha }_v\dot{z}\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x+2{\alpha }_{v}z\mathsf{\Delta }{x}^T{\widehat{J}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }\dot{x}+2{\alpha }_{v}z\mathsf{\Delta }{x}^T{\dot{\widehat{J}}}_{\mathsf{\Omega }}{\widehat{J}}_{\mathsf{\Omega }}^T\mathsf{\Delta }x\\ +2s^T{K}_s^{-1}\widehat{M}{K}_p\dot{s}+s^T{K}_s^{-1}\dot{\widehat{M}}{K}_{p}s\end{array}$$ (59)

$\dot{W}$ is bounded due to Properties 4 and 6. The boundness of (59) illustrates that $W$ is uniformly continuous [38]. Consider the boundness of ${\displaystyle {\int }_0^{t}W(\zeta (r))}dr$, Barbalat’s lemma can be applied. Therefore, $W\to 0$ as $t\to \infty$, which means $\mathsf{\Delta }x\to 0$,$s\to 0$ as $t\to \infty$.

By differentiating (3), it is clear that $\ddot{x}$ is bounded. Hence $\mathsf{\Delta }\ddot{x}$ is bounded too, and $\mathsf{\Delta }\dot{x}$ is uniformly continuous. Since the boundness of $\mathsf{\Delta }x$ is achieved, it can be concluded $\mathsf{\Delta }\dot{x}\to 0$ as $t\to \infty$ by applying Barbalat’s lemma. □

4. Performance Analysis

In order to demonstrate the effectiveness of the proposed controller, numerical simulations have been performed on a UVMS platform. As shown in Figure 4, the UVMS carries a 3 DOFs manipulator and the whole system contains 9 DOFs.

Figure 4.

The configuration of the employed UVMS.

The dynamic parameters of the UVMS are displayed Table 1. The Denavit–Hartenberg table of the manipulator is showed in Table 2.

Table 1.

Main dynamic parameters of the system.

Item	Vehicle	Link1	Link2	Link3
$\mathrm{M}\left(\mathrm{kg}\right)$	82.131	2.603	3.520	3.159
${\mathrm{I}}_{\mathrm{xx}}·(\mathrm{kg}·{\mathrm{m}}^2)$	4.949	0.0262	0	0
${\mathrm{I}}_{\mathrm{yy}}·(\mathrm{kg}·{\mathrm{m}}^2)$	7.362	0.0262	0.1636	0.1636
${\mathrm{I}}_{\mathrm{zz}}·(\mathrm{kg}·{\mathrm{m}}^2)$	8.658	0	0.1636	0.1636

Table 2.

DH parameters of the manipulator.

Joints	${\mathbf{a}}_{\mathit{i}}/\mathbf{m}$	${\mathsf{\alpha }}_{\mathit{i}}/\mathbf{rads}$	${\mathbf{d}}_{\mathit{i}}/\mathbf{m}$	${\mathbf{q}}_{\mathit{i}}/\mathbf{rads}$	${\widehat{\mathit{d}}}_{\mathit{i}}\left(0\right)/\mathbf{m}$
1	0	π/2	0	${\mathrm{q}}_1$	0
2	0	0	0.6	${\mathrm{q}}_2$	0.5
3	0	0	0.3	${\mathrm{q}}_3$	0.5

In simulations, an eye-in-hand camera with parameters ${\mathsf{\alpha }}_{\mathrm{u}}=50$ pixels, ${\mathsf{\alpha }}_{\mathrm{v}}=50$ pixels, ${\mathrm{u}}_0=200$ pixels, ${\mathrm{v}}_0=100$ pixels is employed. Parameters ${\epsilon }_{\mathrm{cam},\mathrm{ee}}^{\mathrm{cam}}={(0.1,0.2,0.1)}^{\mathrm{T}}$ m and ${\epsilon }_{\mathrm{V},0}^{\mathrm{V}}={(0,0,1.5)}^{\mathrm{T}}$ m are employed to describe the kinematic relationships. The configuration of the manipulator follows Table 2. The target is fixed at the point ${\epsilon }_{\mathrm{I},\mathrm{fe}}^{\mathrm{I}}={(2.5,-6,5)}^{\mathrm{T}}$ m w.r.t. the inertial frame.

We set initial values as follows: ${\hat{\mathsf{\alpha }}}_{\mathrm{u}}\left(0\right)=35$ pixels, ${\hat{\mathsf{\alpha }}}_{\mathrm{v}}(0)=75$ pixels, ${\hat{\mathsf{\alpha }}}_0(0)=170$ pixels, ${\hat{\mathsf{\alpha }}}_0(0)=210$ pixels, ${\widehat{\epsilon }}_{\mathrm{cam},\mathrm{ee}}^{\mathrm{cam}}\left(0\right)={(0.2,0.2,0.2)}^{\mathrm{T}}$ m, ${\widehat{\epsilon }}_{\mathrm{V},0}^{\mathrm{V}}\left(0\right)={(0.1,1.4,1)}^{\mathrm{T}}$ m, ${\widehat{\epsilon }}_{\mathrm{I},\mathrm{fe}}^{\mathrm{I}}\left(0\right)={(2.5,-4,2)}^{\mathrm{T}}$ m. The initial configuration of the manipulator follows the last column in Table 2.

As we noted before, these uncertainties can be represented in three constant vectors $\mathsf{\Delta }{a}_{\mathrm{z}}$, $\mathsf{\Delta }{a}_{\mathrm{zd}}$ and $\mathsf{\Delta }{a}_{\mathrm{zin}}$. Details can be found in the proofs of Properties 1–3.

During the simulations, the vehicle starts at the position (0, 0, 0; 0, 0, 0) (m; rads), and the manipulator starts at the position (0, 0, 0) (rads). The initial point of the feature on the image plane is (216, 230).

In simulations, the controller gains are set as follows:

$${\alpha }_v=4.5\times {10}^{-4},K_s{=10}^{-3},{\kappa }_v=0.01,{\lambda }_{\mathrm{zin}}=0.1,{\lambda }_{\mathrm{zd}}=1\times {10}^{-2},{\lambda }_{\mathrm{z}}=1\times {10}^{-2},$$

$$K_p=18\times \mathrm{diag}\left\{10,10,10,15,15,15,5,5,5\right\},$$

$$W=\left[\begin{array}{cc}{0}_{9\times 9}& I_9\\ 0_{9\times 9}& 0_{9\times 9}\end{array}\right],E=\left[\begin{array}{c}{0}_{9\times 9}\\ I_9\end{array}\right],L=[\begin{array}{cc}{I}_9& 0_{9\times 9}\end{array}],p=[\begin{array}{c}0.1\dot{\zeta }\\ 0.2\dot{\zeta }\end{array}],l=[\begin{array}{c}0.1I_9\\ 0.2I_9\end{array}],{\widehat{\gamma }}_2(0)=I_{9\times 1}$$

The desired trajectory is given as:

$$\left[\begin{array}{c}{u}_{\mathrm{d}}\\ v_{\mathrm{d}}\end{array}\right]=\left[\begin{array}{c}300+50\sin (0.1\mathrm{t})\\ 200+50\cos (0.1t)\end{array}\right]$$

The proposed scheme is utilized to track the desired circle trajectory in 70 s.

Figure 5a,b show the actual trajectory and the desired trajectory of the feature point. Figure 5c describes the error trajectory during the tracking routine. Figure 5d illustrates the velocity tracking performance under the proposed method. Time histories of vehicle position, vehicle orientation, and manipulator position can be found in Figure 5e–g. The initial state and the final state can be found in Figure 5h,i.

Figure 5.

Trajectory tracking performance under the proposed scheme. (a) Position tracking performance on the image plane-$u$; (b) Position tracking performance on the image plane-$v$; (c) Position tracking errors on the image plane; (d) Velocity tracking performance on the image plane; (e) Time histories of vehicle position; (f) Time histories of vehicle orientation; (g) Time histories of joint position; (h) The initial state of the UVMS; (i) The final state of the UVMS.

As expectation, rapid convergences have been achieved in both position and velocity tracking performances.

From (18) and (21), one can obtain the connection between $a_{\mathrm{zd}}$ and $\dot{z}$, $a_{\mathrm{z}}$ and $z$. Since the dimensions of $a_{\mathrm{z}}$ and $a_{\mathrm{zd}}$ are all too large, therefore we choose to display the estimation of feature depth and its derivative to reflect the adaption process, the results can be found in Figure 6 and Figure 7.

Figure 6.

Actual depth and estimated depth of the feature.

Figure 7.

Actual depth derivative and estimated depth derivative of the feature.

From (8), we can define following vector:

$${[\alpha ,\beta ]}^T=J_{\mathrm{zin}}(\zeta )\dot{\zeta }=Y_{\mathrm{zin}}(\zeta ,\dot{\zeta })a_{\mathrm{zin}}$$

And its estimation follows:

$${[\widehat{\alpha },\widehat{\beta }]}^T=Y_{\mathrm{zin}}(\zeta ,\dot{\zeta }){\widehat{a}}_{\mathrm{zin}}$$

According to the same reason, we choose to exhibit the actual and estimated values of $\alpha$ and $\beta$ to illustrate the adaption process of $a_{\mathrm{zin}}$. As Figure 8 illustrates, estimation errors stay bounded.

Figure 8.

Adaption process on $a_{\mathrm{zin}}$. (a) Visual tracking performance on $\alpha$; (b) Visual tracking performance on $\beta$.

From stability analysis in previous section, $\mathsf{\Delta }{a}_{\mathrm{z}}$, $\mathsf{\Delta }{a}_{\mathrm{zd}}$ and $\mathsf{\Delta }{a}_{\mathrm{zin}}$ are bounded. Therefore, it is reasonable that estimation errors represented in Figure 6, Figure 7 and Figure 8 are bounded.

Compared with the reference velocity term (33), the proposed method with (34) provide restoring moments optimization by using the GPM. Figure 9 illustrates the two-norm of restoring moments based on (33) and (34), respectively. RMS values of tracking errors based on (33) and (34) can be found in Table 3.

Figure 9.

Two-norm of the restoring moments under different reference velocity terms.

Table 3.

Tracking error RMS values.

RMS Error	Position Tracking Error	Velocity Tracking Error
With the GPM	2.2198	2.8284
Without the GPM	1.9489	2.4959

Because the initial error is relatively large, we calculate RMS values from the time point t = 4 s to the end of the simulation. From Table 3, the controller under reference velocity from (33) provides slightly better results in both position tracking and velocity tracking than the one under (34). However, from Figure 9, the results obtained under the GPM from (34) substantially reduce the influence of the restoring moments compared with the one under (33). Considering all these factors, the scheme with the GPM provides better overall performances.

In addition, we compare he tracking performances between the proposed method and the one but without the HODO. As showed in Figure 10, the proposed scheme achieves accurate and smooth results compared with the other one.

Figure 10.

Trajectory tracking performance. (a) with the HODO; and, (b) without the HODO.

From theoretical analysis in previous section, the HODO achieves BIBO stable, rather than asymptotical stable. But in simulations, the HODO still shows good performances in estimating disturbances in all DOFs. The results can be found in Figure 11. Disturbance estimation error RMS values are given in Table 4.

Figure 11.

Disturbance estimations on: (a) vehicle-X (N); (b) vehicle-Y (N); (c) vehicle-Z (N); (d) vehicle-X (N·m); (e) vehicle-Y (N·m); (f) vehicle-Z (N·m); (g) manipulator-Joint 1 (N·m); (h) manipulator-Joint 2 (N·m); (i) manipulator-Joint 3 (N·m).

Table 4.

Disturbance estimation error RMS values.

RMS Error	X	Y	Z	K	M	N	Joint-1	Joint-2	Joint-3
Disturbance estimation error	4.9847	3.8484	3.7432	1.3211	2.7874	0.6902	0.7359	1.0303	0.1255

5. Conclusions

In this paper, an uncalibrated visual servoing scheme is proposed for the UVMS with an eye-in-hand configuration camera. Parameters of the vision sensor, kinematics of the UVMS and the target position are supposed to be unknown. Firstly, we introduce a linear separation method to collect kinematics uncertainties into vectors, and this method can be utilized in other free-floating based articulated manipulators. Then, to deal with these kinematics uncertainties, novel adaptive laws are proposed to estimate these vectors, the GPM is utilized to optimize the restoring moments of the system. Moreover, a high order disturbance observer is presented to estimate and compensate lumped disturbances. A novel composite controller is addressed in order to realize the convergence of image errors. The stability of the closed-loop system is proved by utilizing Lyapunov theory. Finally, trajectory tracking simulations based on a 9 -DOFs UVMS are carried out to test the proposed scheme under uncertainties and disturbances.

Author Contributions

Conceptualization, investigation, software and writing original draft, J.L.; project administration, supervision and writing -review and editing, H.H.; data curation, Y.X. and H.W.; formal analyzation, J.L., Y.X. and H.W.; methodology, L.W. and J.L. All authors are credited for their contribution on the writing and edition of the presented manuscript.

Funding

This work is supported by National Nature Science Foundation of China (No. 61633009, 51579053, 51209050,51779052, 51779059) and the Field Fund (No. 61403120409) of the 13th Five-Year Plan for Equipment Pre-research Fund, and also funded by the Key Basic Research Project of “Shanghai Science and Technology Innovation Plan” (No.15JC1403300).

Conflicts of Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.