Design and integration of a parallel, soft robotic end-effector for extracorporeal ultrasound

Abstract

Objective

In this work we address limitations in state-of-the-art ultrasound robots by designing and integrating a novel soft robotic system for ultrasound imaging. It employs the inherent qualities of soft fluidic actuators to establish safe, adaptable interaction between ultrasound probe and patient.

Methods

We acquire clinical data to determine the movement ranges and force levels required in prenatal foetal ultrasound imaging and design the soft robotic end-effector accordingly. We verify its mechanical characteristics, derive and validate a kinetostatic model and demonstrate controllability and imaging capabilities on an ultrasound phantom.

Results

The soft robot exhibits the desired stiffness characteristics and is able to reach 100% of the required workspace when no external force is present, and 95% of the workspace when considering its compliance. The model can accurately predict the end-effector pose with a mean error of 1.18 ± 0.29 mm in position and 0.92 ± 0.47° in orientation. The derived controller is, with an average position error of 0.39mm, able to track a target pose efficiently without and with externally applied loads. Ultrasound images acquired with the system are of equally good quality compared to a manual sonographer scan.

Conclusion

The system is able to withstand loads commonly applied during foetal ultrasound scans and remains controllable with a motion range similar to manual scanning. Significance: The proposed soft robot presents a safe, cost-effective solution to offloading sonographers in day-to-day scanning routines. The design and modelling paradigms are greatly generalizable and particularly suitable for designing soft robots for physical interaction tasks.

I. Introduction

It is commonly accepted that sonographers are exposed to an increased risk in repetitive strain injury [1], [2], [3]. A representative study amongst diagnostic medical sonographers and vascular technologists indicates that a significant majority of sonographers experience pain while performing ultrasound scans [4]. This suggests a high demand to improve ergonomics and offload sonographers during clinical scan procedures. Recent investigations show that besides diagnostic sonography, there is an increased demand for intraoperative transthoracic [5], [6] and transoesophegal [7] ultrasound imaging, particularly for cardiac and lung procedures. Sonographers performing intraoperative ultrasound in for example cardiac catheterization procedures have therefore presumably an increased risk of radiation exposure [8].

Automating diagnostic and intraoperative ultrasound procedures through robot-guidance or -assistance can help address the aforementioned problems and lay the groundwork for more intelligent image acquisition. Robotic ultrasound guidance has found particular application in procedures involving steering orthopaedic [9] or minimally-invasive surgical tools [10] and biopsy needles [11]. Various robotic hardware solutions have been proposed. Researchers have adopted robotic platforms originally aimed at collaborative scenarios in industrial settings, such as Universal Robot's UR-series [12], [13] or the KUKA LWR [9] and LBR iiwa [14], [15]. A commercial robotic manipulator has been released (LBR Med, KUKA AG, Augsburg, Germany) which is suitable for use in clinical environments due to its conformity with medical device safety (ISO 60601) and medical software regulations (ISO 62304). Current research suggests that such robots can be applied in diagnostics to autonomously perform aorta measurements [16], in combination with previously acquired MRI scans to autonomously find standard view-planes [17] and in intraoperative procedures to autonomously track surgical tools [18], amongst others. Whilst such robotic platforms allow for great flexibility through a large workspace and high manipulability, the use of large-scale robotic manipulators can pose various disadvantages for clinical integration. Diagnostic ultrasound scans are divided into their respective body area of interest. For an individual procedure such as a lower abdominal ultrasound scan, a robotic system is therefore only required to achieve a workspace to cover a fraction of the human body. This yields that common robotic manipulators could be oversized for such applications, which unnecessarily poses risks to patient safety. Despite high degrees of electrical safety, a mechanical system with a high mass can potentially be more dangerous [19].

To address this issue, researchers developed customized solutions which are tailored to the application-specific requirements of diagnostic and interventional sonography. Researchers [20], [21], [22] have proposed a mechanism which achieves a high degree of probe manipulability and safety. The robot actuation has been moved to the base of the system, thus minimizing its size and weight. Other systems have been developed which separate the probe positioning into two stages: approximate probe placement and finer view-plane adjustments. The first can be achieved by a passive positioning mechanism, which is operated by a clinician, while the latter is obtained with an active end-effector. A system based on cables which are driven by McKibben actuators has been proposed [23]. The antagonistic configuration of the cables is employed to position the ultrasound probe on a patient. The system is tele-operated by a sonographer. Researchers from Waseda University first proposed this concept and corresponding design in [24], in which the end-effector is driven through a parallel mechanism. Similarly, a consortium of researchers have developed a system with active end-effector with the aim of remote tele-diagnosis [25], [26], [27]. The system has since been trialled for remote scans [28] and translated to a commercial product (MELODY, AdEchoTech, Naveil, France). Despite the scanning being performed remotely, the design of the system suggests, however, that the assisting operator is still required to apply the necessary force to maintain a stable contact.

Maintaining stable mechanical coupling between ultrasound probe and patient tissue is of paramount importance for ensuring a high-quality image. Approaches to achieve this involve controlling the contact force directly or establishing an elastic contact between the position-controlled device and the patient. While the first has been researched extensively [29], [30] and can be commonly found in various forms of industrial applications, the latter has found more attention in recent years due to an increased demand in cost effective force control and -limiting solutions for human robot collaboration tasks [31], [32]. Series-elastic actuators have been developed to provide passive compliance in actuated robotic joints [33]. While providing a degree of compliance, this has the disadvantage that a collision or undesired contact in a direction other than the joint axis cannot be compensated for. We have trialled safety clutches for the use in ultrasound robots which exhibit compliant behaviour once disengaged through an excess force [34], [35], [36]. This, however, renders the system uncontrollable and requires reengaging the clutch mechanism for further operation. In this work, we make use of an elastic soft robotic system, which is aimed at overcoming aforementioned limitations. Soft robotics technologies have opened up new design paradigms for robotic systems through the use of elastic and deformable materials and structures [37], [38]. Soft robotics systems are commonly designed to interact with or conform to environmental contacts. This allows soft robotic manipulators to exhibit highly dexterous manoeuvrability in for example surgical [39], [40], [41] or search and rescue operations [42]. In these scenarios, however, soft robots are not applied to tasks which require significant loadbearing capabilities, predominantly due to their low stiffness. To bridge the trade-off between manoeuvrability and stiffness, research has been driven towards systems with variable stiffness capabilities. A comprehensive overview of stiffening technologies is given in [43]. For applications in which softness is desired, high loadings are demanded and stiffening mechanisms are not suitable, soft robotic systems tend to be combined with external constraints to ensure structural integrity. This is commonly found in exoskeleton research and rehabilitation robotics. Examples include full body, soft exosuits [44], lower limb exoskeletons [45] and hand exoskeletons for post-stroke rehabilitation [46], [47].

In our previous work, we identified the advantages of soft robotics technology in ultrasound interaction tasks compared to rigid state-of-the-art robots and showed an initial proof-of-concept of a parallel soft robotic end-effector with the right characteristics for medical ultrasound tasks [48]. We now derive a novel soft robotic end-effector which is capable of safely acquiring standard views in extracorporeal diagnostic foetal ultrasound (US). We select foetal US as an initial application due to its high demands to robot safety. We evaluate the performance of our system with respect to derived specifications and show that the proposed system is capable of acquiring a set of standard view-plane required for the assessment of the foetus. The robot utilizes linear soft fluidic actuators (SFAs) which are arranged in parallel around the ultrasound probe to provide high axial loadbearing capabilities and high lateral compliance, thus enabling adaptability and safety in the patient interaction. The individual contributions of this study are:

• Clinical investigation to determine workspace and force requirements for view-plane adjustments in foetal diagnostic ultrasound imaging.

• Design and verification of a soft robotic end-effector which satisfies the derived clinical requirements in workspace and force. It employs robust linear soft fluidic actuators, for which a novel injection-based fabrication is derived, and undesired twist is prevented through a mesh constraint.

• Definition and validation of a lumped stiffness model to describe the motion of the soft robotic end-effector in the absence and presence of external loading.

The controllability and imaging capabilities of the integrated system are validated in position control and US phantom experiments respectively.

The paper is structured in the following way. In Section II-A the system requirements are determined, and the robot design is introduced. Based on the design of the system, Section II-B derives a kinetostatic model. Methodologies for the actuation and control of the system are presented in Section II-C. In Section III the mechanical properties of the system and its workspace are evaluated. Results are presented in section IV. The proposed model is validated and the position controller performance, as well as the imaging capabilities of the system, are assessed.

II. Methods

Prenatal foetal ultrasound is a routine diagnostic procedure for pregnant women to determine birth defects and abnormalities in the foetus. Common checks include measuring the foetus' biparietal diameter (BPD), its head and abdominal circumferences (HC and AC) as well as its femur length (FL) [49].

In this work we focus on obtaining HC, AC and FL standard view-planes. We establish the clinical requirements to the contact force and movement range of the ultrasound probe for bespoke application and derive a suitable design for a soft robotic end-effector (SEE).

A. Design

1). Clinical data acquisition and processing

Pregnant women between 18 to 24 weeks of gestation underwent research ultrasound scans at St Thomas' Hospital (Study title: Intelligent Fetal Imaging and Diagnosis (iFIND)-2: Further Ultrasound and MR Imaging, Study reference: 14/L0/1806). Trained sonographers performed the foetal ultrasound scan using a standard ultrasound probe (X6-1, Philips, Amsterdam, Netherlands) which is connected to an ultrasound scanner (EPIQ7, Philips, Amsterdam, Netherlands). The probe was placed in a holder as detailed in [50]. This holder incorporated an electromagnetic (EM) tracking sensor (Aurora, NDI, Ontario, Canada) and six axis force-torque sensor (Nano 17, ATI, Apex, USA), which allowed measurements of the position and orientation of the probe, and the force applied at the probe face to be measured throughout the scan. The recorded tracking and force data of six patients were analysed by extracting time ranges during which standard fetal anomaly views were imaged. These included HC, AC and FL views. Each time range consisted of the few seconds when the sonographer had placed the probe in the correct anatomical region and was adjusting the probe to find the ideal view. For each view the tracking data were analysed to find the range of positions and orientations in the three axes separately. The X and Y axes show movement in the horizontal plane of the scanning bed (left to right on the patient, and foot to head, respectively), and the Z axis shows vertical movement. Orientation ranges are given in probe coordinates, with yaw showing axial rotation, pitch showing elevational tilting out of the image plane, and roll showing in-plane lateral tilting. Forces were analysed by dividing the measured force vector into normal and tangential components applied to the surface. The local surface angle was determined at each measurement by fitting an ellipsoidal shape to the tracking data of the scan. The 95th percentile of the forces measured within a time range gives an indication of the maximum force that must be applied by the probe.

2). Mechanism requirements and synthesis

Following the results of the clinical data analysis, it is found that the soft robotic end-effector must at least satisfy the following requirements

• Be able to withstand mean axial and transversal contact forces of 8.01N and 4.42N without significant deterioration of the imaging view.

• Achieve an axial extension along Z of 5.22mm and transversal translations in X and Y of 7.75mm.

• Achieve rotations of 5.08° around X and Y.

To maintain a high degree of safety when interacting with the device, the SEE should furthermore comprise of a low transversal stiffness. This allows both the operating clinician and patient to manually displace the probe in case of discomfort.

As the investigated system is compliant, its deflection has to be considered when determining if a position is achievable. Taking into account normal and tangential forces applied during the scanning, the system must satisfy the following conditions

$${\delta }_{SEE}\ge {\delta }_{req}+{\delta }_f\text{with}{\delta }_f={\mathit{K}}_{\min }^{-1}{\mathit{\text{f}}}_{req}$$

Where δ_f is a deformation induced by external forces, f_req is a vector of the required forces and K_min is the minimum system stiffness throughout the workspace. δ_req and δ_SEE are vectors of the required and achievable translations respectively. As only tip forces are considered in this work, tilting effects induced by external moments at the SEE tip are ignored and forces are assumed to only affect the tip position. A soft robotic design based on soft fluidic actuators (SFAs), which have previously been presented in [48], is proposed. It is comprised of two rigid platforms which serve as base and transducer holder respectively. The platforms are connected through a set of three soft fluidic actuators which are arranged in a parallel fashion at 120° intervals. To allow for sufficient space for the ultrasound transducer cable, the actuators are tilted at an angle of 15°. An overview of the design is shown in Fig. 2a). Whilst a rigid mechanism of such configuration would be over-constrained and thus unable to move, the elasticity of the SFAs allows the SEE to perform bending (coupled translation and rotation) and axial extension motions.

Fig. 2.

Proposed design of the soft robotic end-effector (a) and workflow (b) for obtaining a desired view through manual placement in the approximate region of interest (i) and active steering of the probe towards the desired view-plane (ii).

As the SFAs are tilted, axial extension causes the SFAs to bend into an S-shaped configuration. This allows for the SEE to be axially compliant whilst exhibiting a high degree of load-bearing capabilities, which is further investigated in Section III-B. Furthermore, curving into an S-shaped configuration eliminates the possibility of unstable buckling to occur in the SFAs, as shown in Section IV-C.

A common problem in such a proposed soft robotic system is the low stiffness along its twist axis. To improve the stability of the system against twist deformations, a nylon fibre mesh is attached to base and transducer platforms, which acts as a mechanical constraint between the two. To reduce unwanted buckling behaviour, crimps can be added to the mesh by deforming and heat-treating it. Examples of uncrimped and crimped meshes are shown in Fig. 3. Thus, axial rotation of the ultrasound transducer is not considered in this study, as it could be added by simply applying a rotating mechanism to the base of the SEE, which would function as a stiff rotational axis in conjunction with the mesh constraint.

Fig. 3.

Braided nylon mesh uncrimped (a) and crimped (b).

The workflow of imaging using the SEE is shown in Fig. 2b). Once the SEE is manually placed in the approximate area of the target view using a passive positioning arm, it is fixed on the patient. The ultrasound probe is then actively steered either in a tele-operated manner by a sonographer or in an autonomous fashion using pose or image feedback. As the loadbearing is achieved by the SEE, contact forces the sonog-rapher is required to apply are minimized, which presumably has an impact on the ergonomics of the sonographer.

B. Kinetostatic modelling

To determine the ultrasound probe pose under internal fluid volume variation and external loading a kinetostatic model is derived according to [51]. A free body diagram of the model is shown in Fig. 4. In the following, a vector denoted as w_f represents a 6 degree of freedom wrench in an arbitrary frame f such that ${\mathit{\text{w}}}_f={[F_x^f,F_y^f,F_z^f,M_x^f,M_y^f,M_z^f]}^T$ with forces F and moments M. Similarly, τ_f denotes a reaction wrench in the local SFA frame, which is of the same form as w_f. Vectors noted as δx_f indicate infinitesimally small displacements in frame f of the form $\delta {\mathit{\text{x}}}_f={[u_x^f,u_y^f,u_z^f,v_x^f,v_y^f,v_z^f]}^T$ with translations u and rotations v.

Fig. 4.

Free body diagram of SEE model definition

Let w_ext be a vector of forces and moments applied to the tip of the ultrasound transducer. Under static equilibrium conditions, the following holds for a single actuator

$${\mathit{\text{w}}}_{ext}={\mathit{\text{w}}}_{\theta }+{\mathit{\text{w}}}_V$$ (1)

Where w_θ is the wrench caused by the elastic deformation of the SFA and w_V is the reaction wrench caused by the constrained hydraulic chamber. Both are expressed in the tip frame of the system. The tip wrenches w_θ and w_V can be expressed relative to their local frames by

$$\begin{array}{l}{\mathit{\text{w}}}_{\theta }={\mathit{\text{J}}}_{\theta }(\mathit{\text{x}}){\mathit{\text{τ}}}_{\theta }\\ {\mathit{\text{w}}}_V={\mathit{\text{J}}}_V(\mathit{\text{x}}){\tau }_V\end{array}$$ (2)

Where τ_θ is a vector of local reaction forces and moments caused by the SFA deformation and τ_V is the uniaxial reaction force of the volumetric constraint in the actuator. The matrices J_θ(x) and J_V(x) are defined by

$$\begin{array}{l}{\mathit{\text{J}}}_{\theta }(\mathit{\text{x}})=\left[\begin{array}{cc}\mathit{R}(\mathit{\text{x}})& 0\\ 0& \mathit{R}(\mathit{\text{x}})\end{array}\right]\mathit{A}\mathit{d}\\ {\mathit{\text{J}}}_V(\mathit{\text{x}})=\left[\begin{array}{cc}\mathit{R}(\mathit{\text{x}})& 0\\ 0& \mathit{R}(\mathit{\text{x}})\end{array}\right]\mathit{A}{\mathit{d}}_z=\left[\begin{array}{cc}\mathit{R}(\mathit{\text{x}})& 0\\ 0& \mathit{R}(\mathit{\text{x}})\end{array}\right]\widehat{\mathit{\text{H}}}\end{array}$$ (3)

R(x) is the rotation matrix of the current tip deflection. Matrix Ad is the wrench transformation matrix relating the local SFA frame to the tip frame by

$$\mathit{A}\mathit{d}=\left[\begin{array}{cc}{\mathit{R}}_0& 0\\ {\mathit{\text{D}}}_0{\mathit{R}}_0& {\mathit{R}}_0\end{array}\right]$$ (4)

Where R ₀ is the spatial rotation of the respective frame and D ₀ is the cross-product matrix with the translation vector d ₀ = [d_x, d_y, d_z]. Ĥ is for a single SFA a 6x1 vector containing the third column of Ad.

Considering the elastic behaviour of the SFA, its reaction force τ_θ caused by an infinitesimally small, local displacement δx_θ can be written as

$${\mathit{\text{τ}}}_{\theta }={\mathit{\text{K}}}_{\theta }\delta {\mathit{\text{x}}}_{\theta }$$ (5)

Where the SFA stiffness K_θ is defined as a Timoshenko beam element with

$${\mathit{\text{K}}}_{\theta }=\left[\begin{array}{cccccc}\frac{12EI}{(1+\text{Φ})L^3}& 0& 0& 0& \frac{6EI}{(1+\text{Φ})L^2}& 0\\ 0& \frac{12EI}{(1+\text{Φ})L^3}& 0& \frac{-6EI}{(1+\text{Φ})L^2}& 0& 0\\ 0& 0& \frac{EA}{L}& 0& 0& 0\\ 0& \frac{-6EI}{(1+\text{Φ})L^2}& 0& \frac{(4+\text{Φ})EI}{(1+\text{Φ})L}& 0& 0\\ \frac{6EI}{(1+\text{Φ})L^2}& 0& 0& 0& \frac{(4+\text{Φ})EI}{(1+\text{Φ})L}& 0\\ 0& 0& 0& 0& 0& \frac{GJ}{L}\end{array}\right]$$

L describes the length of the SFA, A it's cross-sectional area, E its Young's modulus, I the area moment of inertia, G its shear modulus and J the torsion constant. The Timoshenko coefficient Φ is defined as

$$\text{Φ=}\frac{12EI}{\frac{A}{\alpha }G{L}^3}$$

with the Timoshenko coefficient α. An overview of the SFA constants is given in Table I.

Table I. SFA model parameters.

Constant	Value	Description
L	45mm	Initial length
A	π ⋅ l02mm2	Cross-sectional area
a	π ⋅ 6.92mm2 *	Fluid channel area
E	30l.5lkPa **	Young's modulus
I	l200cm4 **	Area moment of interia
G	0.5E	Shear modulus
J	0.5 π ⋅ l04mm4	Torsion constant
α	5/6	Timoshenko coefficient

^*obtained in Section IV-B

^**obtained in Section IV-E

Whilst parameters L and A are obtained from the SFA geometry, the torsion constant of a beam with circular cross-section can be expressed as J = 0.5πr ⁴ and its Timoshenko coefficient is defined as 5/6 [52]. The shear modulus G is approximated as half the Young's Modulus.

For a given SFA volume, the kinematic relationship between an infinitesimal small volume change δV of the SFA and the displacement of the ultrasound tip frame is given by

$$\delta V/a={\mathit{\text{J}}}_V^T\delta {\mathit{\text{x}}}_{tip}$$ (6)

Where a is the cross-sectional area of the fluid actuation channel. The kinematic motion of the tip frame caused by the SFA deflection can be defined as

$$\delta {\mathit{\text{x}}}_{\theta }={\mathit{\text{J}}}_{\theta }^T\delta {\mathit{\text{x}}}_{tip}$$ (7)

Substituting Equation 7 into 5 yields

$${\mathbf{\tau }}_{\theta }={\mathit{\text{K}}}_{\theta }{\mathit{\text{J}}}_{\theta }^T\delta {\mathit{\text{x}}}_{tip}$$ (8)

Applying Equations 3 and 8, the static equilibrium condition in Equation 1 can be written as

$$w_{ext}={\mathit{\text{J}}}_{\theta }{\mathit{\text{K}}}_{\theta }{\mathit{\text{J}}}_{\theta }^T\delta {\mathit{\text{x}}}_{tip}+{\mathit{\text{J}}}_V{\tau }_V$$ (9)

Equation 9 can be combined with the imposed kinematic constraint defined by Equation 6 to a linear equation system of the form

$$\left[\begin{array}{c}{w}_{ext}\\ \delta V/a\end{array}\right]=\left[\begin{array}{cc}{\mathit{\text{J}}}_{\theta }{\mathit{\text{K}}}_{\theta }{\mathit{\text{J}}}_{\theta }^{\mathit{\text{T}}}& {\mathit{\text{J}}}_V\\ {\mathit{\text{J}}}_V^T& \mathbf{0}\end{array}\right]\left[\begin{array}{c}\delta {\mathit{\text{x}}}_{tip}\\ {\tau }_V\end{array}\right]$$ (10)

The deflection of the ultrasound transducer tip and internal reaction of the system can consequently be found through matrix inversion

$$\left[\begin{array}{l}\begin{array}{l}\delta {\mathit{\text{x}}}_{tip}\hfill \\ {\tau }_V\hfill \end{array}\hfill \end{array}\right]={\left[\begin{array}{ll}{\mathit{\text{J}}}_{\theta }{\mathit{\text{K}}}_{\theta }{\mathit{\text{J}}}_{\theta }^T\hfill & {\mathit{\text{J}}}_V\hfill \\ {\mathit{\text{J}}}_V^T\hfill & \mathbf{\text{0}}\hfill \end{array}\right]}^{-1}\left[\begin{array}{l}\begin{array}{l}{w}_{ext}\hfill \\ \delta V/a\hfill \end{array}\hfill \end{array}\right]$$ (11)

The formulation can be expanded to a number of n SFAs by considering a lumped stiffness K in the probe tip frame. As the actuators are aligned in a parallel configuration, it can be defined by

$$\mathit{\text{K}}={{\displaystyle \sum _{i=1}^n{\mathit{\text{J}}}_{\theta }^i{\mathit{\text{K}}}_{\theta }^i{\mathit{\text{K}}}_{\theta }^i}}^T$$ (12)

The matrix J_v is adopted by appending the respective columns of the wrench transformation matrix of actuator i $\mathit{A}{\mathit{d}}_z^i$ to Ĥ such that

$${}^n{\mathit{\text{J}}}_V=\left[\begin{array}{cc}\mathit{R}(\mathit{\text{x}})& 0\\ 0& \mathit{R}(\mathit{\text{x}})\end{array}\right]\left[\mathit{A}{\mathit{\text{d}}}_z^1,\mathit{A}{\mathit{\text{d}}}_z^2,\dots ,\mathit{A}{\mathit{\text{d}}}_z^n\right]$$ (13)

The kinematic constraint relationship then becomes

$$\delta \mathit{\text{V}}/a{=}^n{\mathit{\text{J}}}_V^T{\mathit{\text{x}}}_{tip}$$ (14)

Where δV is an n × l vector of SFA volume changes. Consequently, τ_v is expanded to an n × l vector containing n local reactions in the form τ_v = [τ_v, 1, τ_v, 2 …, τ_v, n]^T.

To account for changes in matrices J_θ and J_v for a given motion, the model is solved numerically by dividing the applied external wrench and induced volume vectors into small increments [Δw_ext, Δ V]^T. After each iteration, R(x) is updated according to the previous tip pose.

For the given number of three SFAs, the update rule for the numerical solution is defined by

$$\left[\begin{array}{c}{\mathit{\text{x}}}_{tip}^k\\ {\mathit{\text{τ}}}_V^k\end{array}\right]=\left[\begin{array}{c}{\mathit{\text{x}}}_{tip}^{k+1}\\ {\mathit{\text{τ}}}_V^{k+1}\end{array}\right]+{\left[\begin{array}{cc}\mathit{\text{K}}& {}^3{\mathit{\text{J}}}_V^k\\ {}^3{\mathit{\text{J}}}_V^{k^T}& 0\end{array}\right]}^{-1}\left[\begin{array}{c}\Delta {\mathit{\text{w}}}_{ext}\\ \Delta \mathit{\text{V}}/a\end{array}\right]$$ (15)

For iteration step k.

C. Actuation and control

The SEE is actuated by inflating respective SFAs with a working fluid. As shown in our previous work [53], we utilize custom hydraulic syringe pumps (Fig. 5b)) which are driven by stepper motors (Nema 17, Pololu Corporation, Las Vegas, USA) to induce volume changes in the SFAs. The pumps are controlled with a microcontroller (Teensy 3.5, PJRC, Sherwood, USA) which communicates via a serial interface with a PC running ROS (Intel Core I7-7700HQ, XPS15 9560, Dell, Texas, USA). The PC generates demand velocities or positions for the microcontroller and solves the previously-defined kinetostatic model to determine the system Jacobian for a given pose. Furthermore, the laptop handles interfaces with peripherals such as a joystick for teleoperation (Spacemouse Compact, 3dconnexion, Monaco) and an electromagnetic tracking (EM) system for closed-loop position control (Aurora, NDI, Ontario, Canada).

Fig. 5.

Actuation unit (a) with syringe pumps (b) and controller system

The linear soft fluidic actuators which are utilized to drive the system have first been conceptualized in our previous work [48]. They are comprised of a silicone rubber body (Dragonskin 10-NV, SmoothOn Inc, Pennsylvania, USA) and stiffer silicone rubber endcaps (SmoothSil 945, SmoothOn Inc, Pennsylvania, USA). A helical constraint is inserted into the silicone to counteract radial expansion of the actuator upon inflation. This, in combination with the stiff endcaps, allows for the actuators to maintain its form and only expand in the direction of actuation. The moulding process of creating SFAs has been significantly improved from our previous work. For the radial constraint an extension spring (Fig. 6(v)) is used. The liquid silicone rubber is injected through an inlet (Fig. 6(ii)) using a syringe instead of being poured into the mould. This has the significant advantage for the user to be able to pre-assemble the mould without having to manually wind the constraint helix, as it has been commonly done in soft fluidic actuators [54]. In combination with the injection of the silicone this could reduce variations in the fabrication process. A drawing of a finished actuator is shown in Fig. 6(vii). The combination of radial constraint and stiff endcaps allows for the actuators to be driven efficiently with a volumetric input without exhibiting a nonlinear relationship between input volume and output length change due to bulging, which is investigated in Section IV-B.

Fig. 6.

Overview of mould components (i)-(vi) and drawing of final SFA (vii)

In this work, two methods for controlling the ultrasound probe pose are investigated. A joystick-based teleoperated open-loop controller is implemented to allow a sonographer to steer the probe according to the acquired ultrasound image stream. For this purpose, the aforementioned joystick is used. The axial motion of the joystick is linked to a translation of the SEE in Z-direction while the two tilt axes of the joystick are mapped to the X- and Y-rotation axes of the SEE. The high-level controller generates syringe pump velocities according to

$${\dot{\mathit{\text{V}}}}_{\mathit{\text{d}}}={\mathit{\text{J}}}_V^T{\mathit{\text{v}}}_{\mathit{\text{c}}\mathit{a}\mathit{r}\mathit{t}}$$ (16)

where ${\dot{V}}_d$ is the desired SFA velocity, ν_cart the target velocity in Cartesian space and J_V^T the actuation matrix of the system which has been derived in Section II-B.

A closed-loop controller is integrated to drive the ultrasound probe tip position according to EM tracker feedback. For this purpose, a high-level trajectory generator continuously updates the demand position for the position controller, which generates in return demand volumes for the three syringe pumps according to the control law

$$\Delta {\mathit{\text{V}}}_{\mathit{\text{d}}}={\mathit{\text{J}}}_V^T\mathit{\text{U}}$$ (17)

Where Δ V_d is the desired change in volume and U the control signal. A linear PI controller of the form

$$\mathit{\text{U}}={\mathit{\text{K}}}_P{\mathit{\text{X}}}_e+{\mathit{\text{K}}}_I{\displaystyle \int {\mathit{\text{X}}}_{e}dt}$$ (18)

is employed, where X_e = X_d−X_c. X_d and X_c are demanded and measured probe tip position respectively. The gain matrices K_P = diag (k_P, k_P, k_P) and K_I = diag (k_I, k_I, k_I) contain the gain constants k_P and k_I, which have been verified experimentally and are defined as $0.3\frac{\text{ml}}{\text{mm}}\text{and}0.03\frac{\text{ml}.s}{\text{mm}}$ respectively. The target points are generated at 2Hz while both the position controller and the kinetostatic model are updated at 30Hz. The low-level step generation for driving the syringe pumps is achieved with an update rate of 6kHz.

III. Experimental validation

A. SFA characterization

Using the three SFAs to control the SEE pose in an open-loop configuration requires the volume-extension relation to be predictable for any given point in time. From the radial mechanical constraint incorporated in the SFA design it is assumed that the relationship between induced volume and SFA length change is linear. To verify this, the extension behaviour of a single SFA is investigated for different working fluid changes using a linear rail setup. The position of the tip of the actuator is equipped with a slider and tracked using a linear potentiometer. Contact friction between the linear bearings and rails is minimized using lubrication and friction forces are therefore neglected in the evaluation of the results. Volume and extension data are tracked and synchronized using ROS.

B. Stiffness characterization

As the SEE is highly compliant, knowledge of its deformability under external loads is required to determine its efficacy to the given task. To verify the structural behaviour of the SEE under contact forces required for the clinical application, the stiffness of the system is characterized with the setup shown in Fig. 7. The SEE is mounted to a base plate and its tip is connected through a force-torque sensor (Gamma, ATI, Apex, USA) to a robot manipulator (UR3, Universal Robots, Odense, Denmark). To determine the stiffness of the SEE in a given direction, the manipulator moves the SEE in said direction and the resulting reaction force is measured. The robot allows for an accurate, repeatable displacement of the SEE in a defined direction, thus isolating the desired DOFs. The payload of the system is with 3kg sufficiently high to withstand the induced reaction forces caused by the elastic deformation of the SEE. The motions are repeated 10 times for each configuration. The linearized relationship between reaction force and manipulator displacement corresponds to the stiffness of the SEE.

Fig. 7.

Experimental setup for stiffness characterization

The mesh reinforcement's effect on the axial twist stiffness is determined by twisting the SEE repeatedly by 10° and measuring the z-axis moment. This is done for a configuration without mesh reinforcement, mesh reinforcement without crimps and mesh reinforcement with crimps.

The directional lateral stiffness is obtained by displacing the SEE tip radially in a defined direction over a distance of 10mm. This is repeated for four inflation levels (25%, 50%, 75% and 100% of the maximum SFA volume) and for directions between 0° and 345° in 15° increments around the z-axis. The axial stiffness which corresponds to each extension is determined by displacing the SEE tip in negative z-direction by 1.5mm for 25% and 50% inflation, and by 2.5mm for 75% and 100% extension.

C. Workspace and repeatability

To verify whether the attainable motions of the SEE satisfy the imposed clinical requirements for the ultrasound probe motion, the workspace of the SEE is mapped for achievable volumetric inputs. The SEE pose is measured using an electromagnetic tracker (6DOF Reference, Aurora, NDI, Ontario, Canada) which is attached to the side of the SEE tip. The pose of the ultrasound probe tip is calculated with the known homogeneous transformation between tracker and tip. The SFA volumes are varied between 0% and 100% in 10% increments and the resulting static tip pose is determined with respect to its deflated state.

The repeatability in positioning the tip of the SEE is determined by repeatedly approaching defined SFA volume states and measuring the tip pose. A set of 6 states is defined and the resultant trajectory is executed 50 times.

D. Model validation

The derived model is validated by comparing the workspace and corresponding SFA volumes to the calculated tip pose of the SEE. For this purpose tip poses are calculated for each configuration achieved in Section III-C and the error between model and measurement is determined.

E. Indentation behaviour

Whilst the abdomen exhibits an increased stiffness with the duration of the pregnancy and thus counteracts indentation of the ultrasound probe, deep tissue indentation in the early weeks can affect the positioning behaviour of the SEE. To verify the effect a soft tissue-like contact has on the SEE, a soft mechanical phantom is created. The cylindrical phantom is moulded from a layer of Ecoflex Gel and a structural layer of Ecoflex 00-30 (SmoothOn Inc, Pennsylvania, USA).

The tip of the SEE is controlled to perform a line trajectory from its negative to positive x-axis limits at 60% inflation. The tip pose is monitored with a magnetic tracker and contact forces between SEE and phantom are measured using aforementioned force sensor at the base of the phantom. The manipulator is used to test for different indentation depths from 0mm to 15mm in 5mm increments.

F. Controllability

To achieve a desired view-plane in the ultrasound image, the probe attached to the SEE needs to be steerable accurately across the patient's body. The controllability of the SEE is verified with the closed-loop position control system described in Section II-C. Target trajectories are defined as isosceles triangles with a base of 12.33mm and height of 10mm. For the tilted trajectory, the triangle is titled about one of its sides by 19°. The trajectory is tested in a planar and tilted configuration and tracked 3 times each.

To determine the controllability under an external load, a stiff silicone rubber patch is created as shown in Fig. 8. The patch is lubricated and positioned with its center at the tip of the SEE. To ensure contact with the patch, an initial axial force of 5N is generated by displacing the patch and running the position controller. This is repeated for planar and tilted configurations, where each trajectory is tracked 3 times.

Fig. 8.

SEE moving in contact with soft rubber patch. The tip pose change with respect to the SEEs origin is highlighted with an arrow.

G. Sonographer-guided teleoperation

The imaging capabilities of an ultrasound transducer guided by the SEE are verified using a prenatal abdominal phantom (SPACE FAN-ST, Kyoto Kagaku, Japan). The SEE is equipped with an abdominal ultrasound probe (X6-1, Philips, Amsterdam, Netherlands) which is connected to an ultrasound scanner (EPIQ7, Philips, Amsterdam, Netherlands). A passive positioning arm (Field Generator Mounting Arm, NDI, Ontario, Canada) is used to manually position the SEE in the region of interest on the phantom. The sonographer uses the provided ultrasound image feedback to steer the SEE with the connected joystick towards a desired view-plane. The target view-planes are manually acquired using a handheld ultrasound probe. An overview of the experimental setup is shown in Fig. 9.

Fig. 9.

Sonographer performing SEE-assisted ultrasound scanning of a prenatal abdominal phantom (iii). The SEE (i) is attached to a passive arm (ii) and manually placed on the phantom. A joystick (iv) is used to manipulate the ultrasound probe under visual guidance of the acquired image (v).

IV. Results

A. Clinical data

The results of the clinical data acquisition are presented in Table II. For each subject the maximum observed motion range in translation and rotation of the ultrasound probe is presented for the HC, AC and FL standard views. The presented forces correspond to the 95th percentile of the occurring normal and tangential force magnitudes. A time series of the probe pose and force data obtained for subject 5 is shown in Fig. 10.

Table II. Range of motion and contact force required to obtain a desired view in foetal ultrasound. Values used to generate the required SEE workspace are marked in blue.

		Max. translation [mm]			Max. rotation [deg]			Force range [N]
Subj.	View	X	Y	Z	Yaw	Pitch	Roll	Normal	Tangential
	HC	8.50	3.41	5.67	6.19	4.98	7.72	13.81	1.92
1	AC	4.49	5.60	2.76	3.44	3.45	3.17	4.10	2.60
	FL	6.03	7.86	4.51	5.51	4.84	2.72	7.44	4.15
	HC	4.95	6.45	2.96	5.74	5.91	10.09	14.09	6.15
2	AC	13.53	6.33	7.53	10.80	5.88	12.90	8.73	1.78
	FL	-	-	-	-	-	-	-	-
	HC	9.73	12.41	6.52	7.26	9.80	4.92	13.27	4.87
3	AC	1.53	9.37	2.60	4.23	3.62	4.10	5.55	1.40
	FL	5.81	6.93	7.96	10.34	3.80	7.78	6.61	2.36
	HC	11.87	8.36	7.02	14.84	4.70	9.39	3.47	3.62
4	AC	2.76	2.31	3.19	0.67	1.04	1.09	4.36	3.59
	FL	4.64	4.51	4.64	1.82	3.55	2.07	3.61	2.27
	HC	13.08	11.82	5.79	14.78	4.96	7.14	8.30	8.94
5	AC	9.77	19.91	10.22	2.83	4.55	2.65	4.46	1.49
	FL	4.11	3.77	2.61	6.75	1.92	3.48	3.59	2.92
	HC	2.49	10.18	7.55	7.01	7.83	4.07	20.77	9.13
6	AC	5.69	8.57	4.22	2.27	3.73	1.20	6.06	7.24
	FL	3.79	3.97	3.02	3.60	2.90	1.90	7.97	10.67
	μ	6.63	7.75	5.22	6.36	4.56	5.08	8.01	4.42
	σ	3.64	4.16	2.23	4.09	2.01	3.37	4.70	2.87
	max	13.53	19.91	10.22	14.84	9.8	12.9	20.77	10.67

Fig. 10.

Time series of probe pose (a) and tip force (b) for subject 5. Data between motions corresponding to standard views HC, AC and FL have been omitted.

For subject 2 only HC and AC views were obtained. Translations and rotations are shown with respect to the patient bed. The normal force is assumed to be acting only in negative probe direction and the tangential force shows the vector magnitude of the tangential forces in X and Y.

To obtain workspace requirements which are compatible with the obtained forces, it is divided into transversal and axial movements and transversal rotations. In this study, axial rotations of the probe are ignored. Workspace requirements for the SEE are consequently obtained by selecting the larger translation between X and Y for the transversal ${\delta }_{req}^{tr}$ and the translation in Z for the axial motion ${\delta }_{req}^{ax},$ thus resulting in a required cylindrical workspace of radius ${\delta }_{req}^{tr}$ and height ${\delta }_{req}^{ax}.$ For the orientation, the required rotation is defined by ${\theta }_{req}^{tr}.$ The mean required workspace from the clinical data is therefore

$$\begin{array}{l}{\delta }_{req}={\left[{\delta }_{req}^{ax},{\delta }_{req}^{tr}\right]}^T={\left[5.22\text{mm},7.75\text{mm}\right]}^T\\ {\theta }_{req}={5.08}^{\circ }\end{array}$$ (19)

Corresponding maximum tilts of pitch and roll are in ranges of ±9.8° and ±12.9°. The maximum occurring normal and tangential forces are 20.77N and ±10.67N respectively.

B. SFA characterization

The results of the SFA characterization are shown in Fig. 11. The hydraulic pressure under SFA inflation and the resulting extension are shown in Fig. 11a) and 11b) respectively. Hysteresis is mainly observable in the fluid pressure. The mean deviation from the centerline between loading and unloading is 3.82±1.63kPa for the pressure and 0.14±0.05mm for the extension across the different inflation cycles. A maximum deviation due to hysteresis is observable in the pressure when inflated to 100% with 9.28kPa and when inflated to 50% at 0.44mm. The volume-extension curve of the SFA can be separated into two regions, a nonlinear (0ml to ≈ 1.25ml) and a linear region (≈1.25ml to 5ml). In the linear region, the relationship can be can be approximated with a first order polynomial as ΔL(ΔV) = 6.61mm/ml − 5.52mm. The interpolation is used to determine the relationship between the SFA length change and the input volume change of the form a = ΔV/ΔL = π.6.9²mm². As the proportion of the nonlinear region compared to the overall extension of the SFA is small, it is ignored for the following investigations. SFAs are therefore assumed to be pre-extended with a volume of 1.25ml.

Fig. 11.

SFA pressure (a) and extension (b) under increasing working fluid volume for different inflation levels.

C. Stiffness

The results of the twist stiffness characterization for each mesh configuration are shown in Table III, where μ and σ are the mean and standard deviation of the twist stiffness K_tw respectively. The application of a nylon mesh helps to significantly stiffen the torsional axis of the system by 184%. A crimped mesh can further improve the torsional stiffness to 299% of its original value.

Table III. Twist stiffness.

	None	Uncrimped	Crimped
μ(Ktw) [Nmm/°]	45.94	84.68	137.37
σ(Ktw) [Nmm/°]	0.70	4.30	2.73

The results of the lateral stiffness characterization under inflation of the SEE are shown in Fig. 12a) in polar coordinates. The radius indicates the magnitude of the stiffness in the given direction.

Fig. 12.

Change in transversal stiffness with the direction of the applied force for different extensions (a). Change in stiffness with extension for axial (i) and transversal stiffness (ii) (b). Change in stiffness with bending for axial (i) and transversal stiffness (ii) (c).

The axial and averaged lateral stiffness of the SEE under axial extension are presented in Fig. 12b). The data are presented alongside their corresponding spline interpolations. Both decrease monotonically with the axial stiffness starting from a maximum of 34.83N/mm and reaching a minimum of 14.41N/mm at 100% extension. The transversal stiffness decreases at a comparable rate from 3.21N/mm at 25% down to 1.51N/mm at 100% extension. The stiffness variation under bending of the SEE is shown in Fig. 12c) with the visualized trends interpolated by splines. Whilst the the transversal stiffness decreases monotonically from 3.15N/mm to 1.77N/mm, the axial stiffness decreases from 21.15N/mm at 0.3° tilt to a minimum of 9.99N/mm at 10° followed by an increase in stiffness to 18.75N/mm at 13.75°. The presented data is employed to determine the minimum stiffness of the system throughout the workspace to infer possiblly occurring tip pose deviations from external forces. It can be seen that the system reaches a minimum axial stiffness of 14.41N/mm and transversal stiffness of 1.51N/mm, both in a straight and fully extended configuration.

Despite high loads along the axial direction of the SEE no discontinuous buckling behaviour of the SFAs is observable. This is demonstrated in Fig. 13. The force-displacement relationships and their corresponding linear interpolations are shown for 0%, 50% and 100% extension and depictions of the SEE at the corresponding maximum loads are presented. Whilst a slight increase in the nonlinearity between force and displacement is observable for 100% extension (the corresponding mean absolute errors between data and linear interpolation are 0.84N, 0.62N and 1.16N for 0%, 50% and 100% extension) no discontinuities are identifiable. The depictions of the deformed SEE show how the forced S-shape bending of the SFAs helps to prevent buckling. An increase in axial force only causes the curvature of the S-bend to increase.

Fig. 13.

Measured compression force of the SEE f_meas at 0% (a), 50% (b) and 100% axial extension with it's corresponding linear interpolation f_lin. For each configuration the compressed SEE is depicted and the SFA centerlines are highlighted.

D. Workspace

The workspace of the SEE in position and orientation is shown in Fig. 14. The figures show the tip pose acquired by the EM tracker for any given SFA configuration. The required workspace in position and orientation, δ_req and θ_req, obtained in Section IV-A from clinical data is projected into the center of the SEE workspace. The deflected workspace δ_f is calculated from the results obtained in Section III-B. It can be seen that the SEE exhibits an minimum transversal stiffness of 1.51N/mm and a minimum axial stiffness of 14.41N/mm at 100% extension. Taking into account the mean external load applied to the tip, a possible additional deflection of

$${\delta }_f={\left[\begin{array}{cc}14.41& 0\\ 0& 1.51\end{array}\right]}^{-1}\left[\begin{array}{c}8.01\\ 4.42\end{array}\right]=\left[\begin{array}{c}0.56\\ 2.95\end{array}\right]$$

Thus, the workspace the SEE is required to achieve extends correspondingly to

$${\widehat{\delta }}_{req}={\delta }_{req}+{\delta }_f=\left[\begin{array}{c}5.78\\ 10.68\end{array}\right]$$ (20)

Whilst in some instances larger motions have to be achieved, the derived values represent a baseline motion range desirable from the SEE.

Fig. 14.

Workspace of the SEE in position (a-b) and orientation (c-d). The required workspace δ_req without and with consideration of the deflected tip pose δ_f are indicated. A cross-section view along the dotted lines shows the coupling between position and orientation in the performed bending motions (e), in which the dashed lines indicate iso-volume lines for V ₂ = V ₃.

To quantify whether the SEE is able to reach the desired workspace, the intersections between requirement and SEE workspace volumes are computed. It can be seen that for the unloaded requirements in translation and rotation, δ_req and θ_req, the SEE can accomplish 100% of the workspace. For the workspace adapted to account for an external force ${\widehat{\mathit{\text{δ}}}}_{\mathit{\text{ref}}},$ the robot achieves 95.18% of the required workspace.

It is shown that a maximum combined lateral deflection of 19.01mm can be reached along the principal plane of SFA₃, which is about 4.5% lower than the maximum transversal motion observed in manual scanning. The maximum extension of the SEE of 22.13mm is reached for a full inflation of all SFAs and exceeds the demanded axial translation of 10.22mm as well as the transversal translation of 19.91mm determined from the clinical data. The maximum tilt of the SEE is reached along the principal plane of SFA₁ with 14.02°, which is ≈ 9% greater than the maximum demanded tilt of 12.9°. A maximum axial torsion of 1.03° occurs. Compared to the tilt ranges in X and Y the twist is significantly lower and will therefore be ignored in the following investigations. The coupling between translation and rotation, the bending, of the SEE upon actuator inflation is shown in Fig. 14e) for a cross-section of the workspace along the central x-z-plane in translation and the corresponding y-axis of the rotational workspace. It can be seen that with the amount of transversal translation, the rotation of the tip increases, whilst axial extension has no effect on the rotation.

The results of the positioning repeatability evaluation are presented in Table IV. The table indicates the mean Euclidean errors in position and orientation with their respective standard deviations from the given pose for the 50 repetitions with respect to the mean pose for the given configuration, μ(x(C_j)). For a configuration C_j, for instance, the Euclidean error ||δ_e|| is computed as

$$\Vert {\delta }_e\Vert ={\displaystyle \sum _{i=1}^{n=50}\frac{\Vert {\mathit{\text{x}}}_i-\mu (\mathit{\text{x}}(C_j))\Vert }{n}}$$ (21)

The pose x_i for a given configuration C_j is obtained by averaging the measured static tip pose over a period 4 seconds. The orientation error ||θ_e|| and both corresponding standard deviations are calculated in the same manner. Whilst it can be seen that the measured accuracy of the SEE is with ≈ 0.1mm in position and 0.05° orientation slightly below the rated accuracy of the EM tracking system (0.48mm and 0.30° RMS [55]), it can be seen that averaging the pose data over 4 seconds reduces noise-related variance in the data. The samples are normally distributed across the workspace and thus the time-averaged mean is assumed to represent the tip pose sufficiently.

Table IV. Repeatability.

Pose	[V1, V2, V3]	||δe||[mm]	||θe||[°]
C 1	[0%, 0%, 0%]	0.07 ± 0.05	0.03 ± 0.02
C 2	[75%, 50%, 75%]	0.10 ± 0.05	0.05 ± 0.02
C 3	[25%, 0%, 100%]	0.07 ± 0.03	0.06 ± 0.03
C 4	[50%, 25%, 0%]	0.08 ± 0.06	0.04 ± 0.02
C 5	[70%, 80%, 25%]	0.09 ± 0.04	0.06 ± 0.03
C 6	[0%, 20%, 70%]	0.11 ± 0.05	0.07 ± 0.03
	μ	0.09	0.05

E. Model validation

The results of the model validation are shown in Fig. 15 and summarized in Table V, where μ refers to the mean error, σ to the standard deviation and max to the maximum error. The estimated workspace of the SEE generated with the kinetostatic model is shown in Fig. 15. The colour of each marker indicates the Euclidean distance between the calculated point and the corresponding measured pose normalized to the maximum error in position and orientation respectively, namely 2.37mm and 2.46°. The Young's modulus of the SFA material E and its area moment of intertia I have been manually tuned to minimize the Euclidean error in position and orientation. The obtained values are shown in Table I.

Fig. 15.

Workspace generated with model in position (a) and orientation (b). The colour indicates the normalized Euclidean error in the given state with respect to the maximum deviation from the model, which is 2.37mm in position and 2.46° in orientation.

Table V. Model validation.

	Displacement [mm]			Tilt [°]
	μ	σ	max	μ	σ	max
ex	-0.81	0.20	1.25	0.04	0.44	1.34
ey	-0.55	0.47	1.60	0.05	0.86	2.26
ez	-0.05	0.50	1.80	-0.13	0.36	1.36
||e||	1.18	0.29	2.37	0.92	0.47	2.47

Overall, the model validation shows with a mean Euclidean error of 1.18 ± 0.29mm in position and 0.92 ± 0.47° in orientation good results in predicting the tip pose under SFA extension.

F. Contact experiment

The motion constraint induced by an indentation contact is investigated. Fig. 16 shows the constraint of the mean X-displacement and Y-tilt for a given motion over 10 repetitions normalized to their respective maximum values of 12.92mm in position and 8.67° in orientation when no external force is present, as well as their corresponding linear interpolations. The transversal force applied by the SEE is measured with the force torque sensor. For both, the displacement and the tilt, the magnitude declines monotonically. Whereas the displacement reaches a minimum at 27.84%, the tilt remains less affected by the lateral force with a minimum of 55.35%. Linearizing the trends yield a decrease of 14.09%/N for the displacement and only 8.56%/N for the tilt.

Fig. 16.

Effect of axial loading on transversal motion

G. Position control

An example of a tracked trajectory with external loading is shown in Fig. 17. The position controller tracks the desired position accurately with marginally larger tracking error around the corners of the triangular path. The quantitative results of the controller evaluation for the three executions are presented in Table VI for both the unloaded and loaded trajectories, where, as in Section IV-E, μ refers to the mean error, σ to the standard deviation and max to the maximum error in the respective direction. The results indicate a higher mean error for the z-direction regardless of the configuration, which is also observable in the visualization above.

Fig. 17.

Example of tracked trajectory under external loading. The normal force is represented with scale of 0.2mm/N

H. Teleoperation and image-acquisition

The images obtained through manual ultrasound probe placement and steering with the SEE are presented in Fig. 18. Anatomical structures of the foetus phantom are clearly visible throughout all images with minor shadowing on the left side of the FL standard view-plane, outside of the region of interest. In both cases, the regions of interest are centered in the image. Moreover, the contrast in the robot-acquired images is similar to the one in the manually-obtained images.

Fig. 18.

Ultrasound images acquired by sonographer (a-c) and SEE (d-f) for HC (a,d), AC (b,e) and FL (c,f) measurements

V. Discussion

In this work we developed a soft robotic ultrasound imaging system to offload sonographers in day-to-day scanning routines. The system addresses the issue of providing a stable contact between the ultrasound probe and patient, which could help improve sonographers' ergonomics particularly with respect to work-related musculoskeletal disorders which arise from stresses induced by repeated manual probe handling. The robot allows for tele-operated scanning and provides a platform for advanced imaging approaches. It is designed in form of an end-effector which is manually positioned in the area of interest and actively steered towards the desired view-plane. Due to its inherent compliance, the SEE is able to maintain contact while exhibiting sufficient axial stiffness to ensure mechanical coupling for the ultrasound image acquisition which is verified by acquiring standard views on a foetal ultrasound phantom.

The system shows with its high axial and low lateral stiffness good applicability to foetal ultrasound scanning. Despite the quick decline of stiffness with axial extension, the SEE is with 14.41N/mm axial stiffness at full extension still capable to apply sufficiently high forces to the patient without significant deformation, which is approximately 1.44mm at maximum axial load of 20.77N. The lower lateral stiffness allows for the system to be adaptable to the contact surface and to be moved away in case of discomfort in the patient whilst being sufficiently high to counteract transversal loads occurring during the intervention. It can be seen that for the fully extended SEE the transversal displacement at a maximum occurring load of 10.67N reaches 7.1mm.

The compliance of the system allows for deformation upon external motion when clamped onto a patient. Thus, the resulting contact force is significantly lower compared to a rigid system. It furthermore exhibits a low mass which could be beneficial in the dynamic safety of the system [19].

If the stiffness in the axial direction of the probe needs to be adjusted or the tip force controlled, the system can be equipped either with a force sensor at the base to estimate tip forces or serve as a sensor itself [56]. While in the first case the tip pose change during the operation needs to be accounted for to accurately determine the external force, either by an accurate model or pose feedback, the second case can make use of the deformable structure of the robot paired with the kinematic constraints induced by the actuation channels to infer the external force.

We have shown that the integration of a braided nylon mesh, which has previously only been used to avoid ballooning in SFAs, can significantly improve the twist stiffness of the SEE to up to three times in comparison to the mesh-free system. The use braided meshes is a highly versatile design approach and shows the potential to become a de facto standard in reinforcing not only soft robotics system but also continuum robots against unwanted external twists induced by contact wrenches, thus enabling such robots for a wider range of applications.

The workspace achieved by the SEE covers without external loading the average translation and rotation motion ranges required to achieve a desired view, as shown from clinical data. Loading the probe with the contact forces measured in clinical scans and assuming the lowest possible stiffness of the system reduces the achieved workspace to about 95.18% of the mean required range. Whilst, for example, the maximum translation of the SEE is at 19.01mm significantly higher than the required deflected motion of 10.68mm, the non-homogeneous shape of the SEE workspace dictates the limitations in covering the required translation range. This limitation could be addressed by adding a linear stage to the base of the SEE to allow for axial translation without sacrificing the softness of the system. Moreover, an axial rotation stage could be added to allow for more complex probe motions.

A high variability in the monitored ultrasound probe motion ranges can be observed across the obtained views and subjects. Whilst, on average, relatively small maximum deflections are observed, in some instances significantly larger motions occur. This is indicated by the high standard deviations in the motion ranges of the respective axes. Further research needs to be conducted into the exact metrics of the ultrasound probe motions and whether the designed system can satisfy those metrics. Additional considerations such as the coupling between different motion axes then need to be accounted for. Another factor in the feasibility of a desired view is the accuracy of the manual placement of the passive positioning arm. If the accuracy is low and the view is out of reach of the end-effector, the passive arm could either be repositioned manually or additional DOFs could be added to the system. More accurate methods should be employed in evaluating the manual probe motions. The use of a percentile is difficult for the given data due to the high variability in the times required to obtain desired views, as seen in the presented time series for the motions of subject 5 in Fig. 10 for example. Thus, a larger scale and more streamlined data acquisition needs to be conducted.

We showed that the combination of SFAs and hydraulic actuation exhibits good properties for the SEE to be driven in an open-loop configuration. The relationship between SFA length and input volume is highly linear and only shows 0.14±0.05mm deviation due to hysteresis, thus allowing for an accurate prediction of the kinematic constraints imposed on the SEE. This compliments the derived kinetostatic model, which is able to accurately predict the SEE tip motion with an accuracy of 1.18mm in position and 0.92° in orientation as a function of the induced working fluid volume. The model deviates more along the boundaries of the workspace, which could be caused by the larger deflection of the SFAs and resultant nonlinearieties caused by the bending of the actuators. This could be addressed by extending the model to a nonlinear approach, as we have for example demonstrated in [56] for a soft continuum robot.

The repeatability lies with 0.1mm in position and 0.05° in orientation slightly below the rated accuracy of the measurement system. As the obtained measurements are expressed relative to a mean pose, averaged over time and normally distributed, it is assumed that these values still represent the true pose well. The high repeatability and should allow for accurate positioning of the SEE in view-plane finding applications.

The system maintains stability and controllability well when in contact with a tissue-like soft silicone rubber patch. We showed that the implemented closed-loop position controller is able to track target trajectories accurately with a mean position error of 0.35mm with only marginally increased errors in the tracking accuracy of 0.44mm when a contact force applied. In scenarios where EM tracking is not available, the ultrasound image could be used to provide pose feedback. This could then employed as a substitute for the position feedback in the closed-loop controller.

The coupling between position and orientation is an obvious limitation in the usability of the design. It can be seen, however, that the mechanical properties of the surface contact greatly affect the coupling behaviour. We have shown that an indenting contact reduces the lateral motion of the ultrasound probe significantly more than the tilt. It can easily be seen that a very stiff coupling in combination with the minimal contact friction caused by the application of ultrasound gel greatly reduces the tilt capabilities of the system while allowing for lateral sliding. It can therefore be assumed that in practice the coupling can be reduced by varying the axial pressure applied to the patient. This is supported by the findings of the tele-operated image acquisition in Section IV-H and will be investigated further in future research.

VI. Conclusion

The SEE design proposed in this work shows a novel approach to applying soft robotics technologies in medical ultrasound imaging. We have shown that under certain conditions the SEE satisfies the requirements imposed by the clinical application. The derived kinetostatic model mimics adequately the behaviour of the physical robot and the integrated system is capable of tracking target trajectories accurately and obtaining high-quality ultrasound images of a prenatal ultrasound phantom. In our future work, we will make use of the hydraulic actuation to integrate a force-controlled system through intrinsic force sensing, as shown in our previous work [56].

Fig. 1.

Soft robotic end-effector (SEE) performing ultrasound scan on abdominal prenatal phantom

Table VI. Position control results.

	Flat - Unloaded			Tilted - Unloaded
	μ	σ	max	μ	σ	max
ex [mm]	0.20	0.19	0.98	0.19	0.19	0.82
ey [mm]	0.25	0.20	1.03	0.27	0.20	0.88
ez [mm]	0.11	0.10	0.52	0.13	0.07	0.35
||e|| | [mm]	0.34	0.29	1.51	0.36	0.28	1.25
	Flat - loaded			Tilted - loaded
	μ	σ	max	μ	σ	max
ex [mm]	0.24	0.23	1.10	0.27	0.28	1.50
ey [mm]	0.33	0.22	1.08	0.32	0.25	1.07
ez [mm]	0.13	0.10	0.56	0.17	0.10	0.65
||e|| [mm]	0.42	0.33	1.64	0.45	0.39	1.95