Design of a pneumatic soft actuator controlled via eye tracking and detection

Abstract

This work describes the control of a pneumatic soft robotic actuator via eye movements. The soft robot is actuated using two supply sources: a vacuum pump and an air supply pump for both negative and positive air supply sources respectively. Two controlled states are presented: the actuation of the vacuum and air pump. Through eye positioning and tracking on the graphical user interface to actuate either pump, a control command is directed to inflate or deflate the pneumatic actuator. The potential of this application is in rehabilitation, whereby eye movements are used to control a rehabilitation-based assistive soft actuator rather than ON/OFF electronics. This is demonstrated in this work using an elbow based rehabilitation soft actuator.

Biomedical Engineering; Electrical Engineering; Data Visualization; Computer Engineering; Robotics; Materials Characterization; Structural Analysis; pneumatic actuator; soft robot; pleated network design; corrugated actuator; eye tracker controlled actuator; eye gaze system for control; tobii tracker 4c with soft actuator

1. Introduction

As a result of the deformable bodies of soft actuators, potential benefits pertaining to compliance and use in a variety of novel applications are possible. Soft robots are able to manoeuvre using the infinite degrees of freedom they possess owing to their characteristic large deformations. Applications of these robots include in hazardous environments, crawling movements [1], muscle like movements [2], [3], hand [4], elbow [5] and shoulder [6] rehabilitation. An appropriate control system comprising both software and hardware components are therefore required to control these various movements experienced by soft actuators.

The control hardware usually used to actuate non pneumatic soft robots includes Dielectric Elastomer Actuators (DEAs) [7], Shape Memory Alloys [8], Hydrogels [9] and Pneumatics [2]. Pneumatic soft actuators are controlled using pistons, cylinders, valves, compressors, air and vacuum pumps. These hardware components are normally linked to electronics and microprocessor boards for automatic control. These electronics interface in turn requires the use of ON/OFF circuit switches to direct the soft actuators to move in a specified format according to the desired motion. When an electronics interface is not provided, mechanical or manual control of the actuators are made possible using the soft actuators which is generally less preferred compared to an electronics controlled interface due to the inability of a mechanical or manual interface to be integrated with sophisticated systems that would improve the control strategy.

With the use of an electronics control, more sophisticated interfaces can be integrated into the control of a soft actuator such as the introduction of biological movements such as hand gestures, voice or eye movements in order to eliminate the use of circuit switches. Few approaches have been carried out in literatures to bypass the use of circuit switches to regulate the movements of soft actuators through electronic control. One approach is with the use of gestures such as swipe, tap, hand close and hand open gestures [10]. These gestures are translated into control actions that move and direct the soft actuator. Gesture sensors such as the leap motion sensor have been used to achieve this [11]. The leap hand trackers have been used such that the movement of the fingers (master device) follows the movement of the slave device (finger-based soft robot) [12]. In another work, specific gestures were extracted from a leap sensor to control the actuation hardware components [13]. Another work made use of EEG as well as virtual reality to direct the movement of a pneumatic soft robot for inflation and deflation cycles [14]. A binocular eye tracking glasses which when worn, enabled the actuator to be directly observed and generate specific motion has been achieved [15], this eliminates the requirement of user interaction with a display screen or other input and output interfaces. The major limitation of this interface is evident as a result of the cumbersome nature of the binoculars which would cause fatigue when worn by a user to control a soft actuator. As part of the attempt to control soft actuators using sophisticated methods, a voice controlled hand rehabilitation device [16] have been produced and analysed to detect user intention for various grasping tasks.

This work involves the use of eye tracking technology to track the location at which the eyes are focused on a computer screen which would in turn be translated into specific control commands that would produce intended motions directed into a soft actuator. Two graphical areas are defined: when the eyes are fixed on the first region, the air supply source is activated to inflate the actuator, when the eyes are fixed on the second region, the vacuum supply source is active to deflate the actuator, when neither condition exist, all the actuation components are deactivated. This application is demonstrated on a wedge-shaped soft robotic actuator with a corrugated structure designed to rotate the elbow joint. In this way, a user can control the motion of the elbow as well as the motion of the soft actuator simply by using eye gaze data from Tobii Eye Tracker 4C. The specific soft robot is first fabricated and characterized; then the integration and control design using eye tracking is carried out.

The novelty of this work is the integration of an eye tracking interface to control a corrugated soft actuator for variation in angular motions. The novelty comes with the application of a non-invasive technique for eye tracking so as to minimize user fatigue. The paper is organised as follows: this section provides recent advances in the integration of soft robots with advanced control interfaces. Section 2 gives background information on the eye tracking and detection interface used. Section 3 provides a detailed procedure for the fabricated soft robot in the form of a wedge which has corrugations on opposite side. In Section 4, the fully integrated system is presented and analysed. Section 5 is the concluding section which provides the conclusion and possible future directions of the work.

2. Eye tracking and detection interface

An eye tracker is a hardware device whose internal component is composed of projectors and sensors that keeps track of eye location on a computer screen [17]. The technology that enables a computing device to determine eye position and gaze point on a display screen is called eye tracking. Though eye tracking has been used in professionally for many years, it is only of recent the technology became more readily available for consumer use [18].

The Tobii Eye Tracker 4C (Figure 1a) is a commercially available eye tracking device [19]. The tracker works with an Illumination module that emits a near infrared light beam which creates reflection patterns on the eyes, these reflections are captured by the cameras in the eye tracker. Then, with filtering and triangulation, the eye tracker determines the gaze point and obtains eye movement data. With near infrared (NIR) light illumination technology, the device operates on a wavelength that is undetectable to the human eye and thus can track the direction of gaze point over an extensive range of conditions such as varying user positions and lighting conditions. The system detects eye movements, enabling gaze to be determined from analysis of the reflection points in conjunction with additional data measured from eye and facial representation. It uses Tobii EyeChip which is a fully embedded EyeCore that provides algorithm processing and illumination control, enabling the system to work independently of the host computer, thereby minimizing power consumption and load on the system CPU [20].

Figure 1.

(a) The Tobii Eye Tracker 4C powered ON via USB connection to a computer. When powered on, the infrared camera and projectors come ON as shown (b) Window showing Tobii eye tracking status: b(i) the eyes position are level b (ii) the user sits with the head tilted b (iii) sitting with left eye closed.

The Tobii eye tracker comprises of infrared (IR) projectors, IR cameras and computer vision algorithms on a DSP core. A pattern of near-infrared light of NIR 850nm is created by the projector on the eyes. The high frame rate cameras take high resolution images and reflection patterns of the eyes. Then, important eye details and reflection patterns are mapped onto computer vision algorithms image-processing algorithms map specific details in the user's eyes and reflection patterns. The consequence of these processing results in the creation of image processing algorithms that accurately calculates the gaze point and position of the eyes on a particular spot on a computer screen. The Tobii Eye Tracker 4C transmits normalized gaze coordinates [21].

Three instances are shown in Figure 1b of a user sited before a computer screen with the positioning of the eye tracker as shown in Figure 1a. In Figure 1b (i), the participant sits directly before a computer screen with the head straight and the eye tracker positioned at the proper level, in Figure 1b(ii), the user has the head tilted, while in Figure 1b(iii) one eye of the user is closed.

3. Soft robot design & characterization

The triangular soft actuator which has a corrugated outward face is entirely made out of silicone rubber. The technique involved in the fabrication process involves a moulding process which comprises of mould fabrication and arrangement, pouring the silicone rubber in liquid form into the mould, allowing it to cure and removal of the moulded silicone.

For the mould fabrication, the design of the mould is made using a 3D drawing software, SolidWorks®. Two types of moulds are required: the main mould and the enclosing mould. The main mould is used to cast the internal structure of the soft actuator while the enclosing mould is used to create an enclosed or air-tight closure for the actuator. The moulds are then 3D printed with ABS plastic using a HP Designjet® printer which is a FDM printer. A sketch of the actuator is shown in Figure 2. A table specifying the parameters of the actuator is shown in Table 1.

Figure 2.

Corrugated triangle actuator sketch.

Table 1.

Parameters of the corrugated triangle actuator for fabrication.

Parameter	Value
$\theta$	${45}^0$
$n_c$	$7$
$h_c$	$120mm$
$l$	$100mm$
$t$	$2.5mm$
$d$	$5mm$

The internal design of the specific soft actuator made consists of a triangular or wedge-shaped structure with corrugations or pleats on the hypotenuse side of the triangle. The result of this specific design is that when the actuator is pressurized from a pneumatic source, the pleats, or ridges are first unwound followed by the stretching of the walls. In other words, given that $d$ is the corrugation height, $n$ is the number of pleats and $l$ is the length of the hypotenuse, when inflation pressure is supplied, the hypotenuse length first unwinds to $l+2nd$, then afterwards begins to stretch further. As a result, a considerable change in angle is observed with a pleated network design compared to when there are no pleats. The variables of the wedge-shaped corrugated module of Figure 2 are indicted in Table 1 together with their respective parameter values.

The produced corrugated actuator is shown in Figure 3. The silicone grade used in its production is the Ecoflex 0030 liquid silicone rubber. The liquid silicone comes in two parts which are mixed together in equal proportion by weight or volume and poured into the fabricated plastic mould. Due to the high surface tension of liquid silicone, bubbles form. These bubbles gradually rise to the top surface of the moulds and are removed manually by blowing them away. The mixture sits in the mould for 4 h, which is a process that allows it to cure at room temperature of $27°C$. After curing, the cured silicone rubber is peeled out of the mould. In order to prevent tear and facilitate smooth removal of silicone rubber from the plastic mould, the mould is first sprayed with lacquer or rubbed with petroleum gel. After removal, the two parts of liquid silicone are mixed and poured into the enclosing moulds; the cured silicone is placed on the liquid silicone so as to produce an air-tight and seamless closure. Silicone rubber tubing are inserted into the actuator by drilling a hole into the body of the actuator, and inserting the tubing therein. Mixed parts of the silicone liquid are then poured in the outer parts of the insertion to block air leakage via the drilled hole.

Figure 3.

Produced 45°^° corrugated triangle actuators for angular movement.

This corrugated design in the outward facing side of the soft actuator is an improvement to the previous work [5] which uses a smooth outward facing side rather than the use of a corrugated structure. In this work, the use of a pleated network (Figure 3) in form of corrugation is employed so that when supplied with pneumatic air pressure, unwinding of pleats first occurs which is accompanied by the stretching of silicone rubber. The advantage of a corrugated design results from the unwinding of the corrugations followed by expansion of the outward face under pneumatic action. As a result of this unique feature, a larger angle range is produced compared to when there are no corrugations.

To assert the suitability of silicone rubber to be used for inflation and deflation cycles, the silicone rubber was first tested for stress and strain measurements which provided an estimate of the actuator's ability to operate under stresses provided by pneumatic action. As shown in Figure 4, the Ecoflex 0030 silicone rubber was found to exhibit a stress-strain relationship consistent with large deformations due to elastic behaviour, hence the choice of silicone rubber.

Figure 4.

Experimental data of stress-strain curve of Ecoflex 0030 silicone rubber. An extensive deformation can be seen under a constant load, which is consistent with elastic behaviour and hence the choice of the silicone rubber for inflation and deflation cycles.

The positive air pressure as a function of time is shown in Figure 5a while negative air pressure as a function of time is represented in Figure 5b. From Figure 5a, the actuator obtains a pressure of 10kPa in 35 s while from Figure 5b, a range of negative air pressure from 0 to -45kPa is obtained between 0 to 15 s. 6V air pressure and vacuum pumps are used as pressure source respectively.

Figure 5.

Air pressure as a function of time for corrugated soft actuators (a) The actuator attains a positive air pressure of 10KPa in 35s (b) The actuator obtains a negative air pressure of $\approx -45$kPa in 15s.

The angle measurement as a function of time for positive air pressure is shown in Figure 6a while angle as a function of time for vacuum pressure actuation is represented in Figure 6b. From Figure 6a, the actuator rotate by a significant amount of $60°$ in 35 s while from Figure 6b, an angle change of $30°$ is observable. The reason for the larger absolute range of motion for positive air pressure compared to negative pressure actuation is due to the corrugations present. Under positive inflation pressure, there is unwinding of the corrugations which lead to a comparable range of motion. In the case of negative pressure actuation, the corrugations hinder the range of movement which results in a limited range of motion.

Figure 6.

Angle measurements as a function of time for air and vacuum pressure for corrugated soft actuators (a) The actuator attains an angle of $60°$ in 35s for positive pressure (b) The actuator obtains an absolute angle deviation of $30°$ in 15s for vacuum actuation.

The angle as a function of positive air pressure is shown in Figure 7a while the angle as a function of negative air pressure is represented in Figure 7b. From Figure 7a, the actuator obtains an angle of $60°$ in 10kPa while from Figure 7b, an angle of $30°$ is obtained at -45kPa. From the results, a considerable range of motion is obtained for air pressure actuation compared to vacuum actuation because the pleated networks design enhanced the range of motion for positive pressure but inhibited the range with respect to vacuum actuation.

Figure 7.

Angle measurements as a function of air and vacuum pressure for corrugated soft actuators (a) The actuator attains an angle of $60°$ at 10kPa positive pressure (b) The actuator obtains an absolute angle deviation of $30°$ at -45kPa vacuum pressure.

In order to verfify the torque output, a dummy joint was fabricated to model a one degree-of-freedom elbow joint. The experimental setup of the torque is shown in Figure 8 comprising of the elbow joint model, load cell and protractor for precise angle measurement A TAL220 straight bar load cell was used to collect torque measurement data. In order to obtain the maximum torque at a given angle, the elbow joint is locked at the given angle while air pressure within the soft actuator is regulated to its operating limit. The maximum torque at the fixed angle is obtained when the air pressure in the wedge-shaped robot reaches its maximum operating pressure.

Figure 8.

Experimental setup for torque measurement. Torque, T = Fd, where F = Force and d is the distance measured from the joint to the line of application of the force.

The torque as a function of positive air pressure is shown in Figure 9a while the torque as a function of negative air pressure is represented in Figure 9b. From Figure 9a, the actuator obtains a positive torque of 2.8Nm in 20kPa while from Figure 9b, a negative torque of 2.5Nm is obtained at -45kPa. From the obtained results, a higher torque is obtained for air pressure actuation compared to vacuum actuation because the pleated networks design enhanced the range of motion for positive pressure and consequently led to an improved torque when compared with actuation through vacuum pressure. Based on this result, both modes of actuation (positive air and vacuum supply) is essential for operating the triangular, pleated modules for optimal performance.

Figure 9.

Torque measurements as a function of positive and vacuum pressure for corrugated soft actuators (a) The actuator attains a maximum torque of 2.8Nm at 20kPa for positive pressure (b) The actuator obtains a maximum torque of 2.5Nm at -45kPa for vacuum pressure.

The intended use of the pleated, triangular actuator is for moving the forearm at the elbow joint such that a human forearm placed on top will move upwards and downwards similar to the motions of extension and flexion of the elbow. As a result, a stable mode of interfacing the actuator with the human forearm becomes essential. The design requirements involved in an interface design for attaching the forearm to the soft actuator includes a comfortable and skin safe material, secure and firm fit, ease of attaching and compatibility with the silicone actuator. The attachment also serves to transfer torque and air pressure distribution from the actuator to the human forearm. In order to meet these criteria, an elastic textile material called Lycra was used. The material, being elastic in nature interfaces seamlessly with silicone rubber. The manufacturing process for integrating the soft actuator with Lycra involved mixing both parts of liquid silicone rubber and pouring on the adjoining layers of the textile material and triangle actuator for the best fit. A firm fit was ensured by sewing Velcro straps onto the Lycra which secures the forearm in place to prevent slip.

Figure 10 shows the pleated actuator together with the designed attachment and a user wearing it. As a result of the pleated structure, the actuator can move the forearm placed on it at a significantly range of angle but with a much smaller torque. Therefore, the mode of operation involved fixing the actuator on a flat surface or table top, a user places the forearm on in and actuation modes are activated for positive air pressure and negative air pressure cycles which extends and flexes the elbow joint appropriately. A generic attachment was fabricated so that user experiments can be conducted with a single fabricated actuator. Participant experiments were approved by the ethical approval committee at the School of Biological Sciences, University of Reading. The ethical approval number is SBS16-17 15. All subjects involved in the study provided informed written consent.

Figure 10.

The soft, pleated actuator in use by a participanta) the forearm rests on the actuator while no air pressure is supplied b) the actuator moves the forearm upwards with the application of positive supply pressure. This generic design allows for preliminary studies that validates the integrated system to be carried out.

Also, how well the system works for multiple runs was determined by measuring the angle-pressure relationship for one user experiment so as to test for the repeatability of the system under the same loading condition. Figure 11 shows the angle-pressure measurements for two multiple runs. An angle range of 0^°–40^° is observed for a pressure range of 0–10KPa for both multiple runs. This result indicates that the corrugated soft actuator is able to produce same results when run multiple times. The fluctuations observed are as a result of the positive air pressure inside the soft silicone robot. These air pressure fluctuations are due to the characteristic property of large deformations of silicone rubber, thee large fluctuations observed are characteristic of damping due to energy dissipation when the elastic walls are stretched [22].

Figure 11.

Angle of the soft, pleated robot as a function of air pressure in kPa for two multiple runs for a single user. An angle range of 0^°–40^° is observed for a pressure range of 0–10KPa for both multiple runs.

4. Control design of soft actuator with eye tracking

The soft actuator was integrated with the eye tracker through the USB interface of a computer. The electronic control of the soft actuator is the same as presented in presented in previous works [2], [10], [23]. The electronic control links to a PC through a USB interface. Control commands are directed from the eye tracking software to the soft actuator using the serial interface port.

The electronics board together with the soft actuator is shown in Figure 12 which indicates the connection arrangement of pumps that connects the corrugated actuator with the electronic control circuit.

Figure 12.

The arrangement of hardware components consisting of the pleated robot (with Velcro strap), electronics circuit and pressure pumps.

A Graphical User Interface is designed that includes two distinct square areas as indicated in Figure 13a. Each area is clearly marked indicating whether to turn ON either the air pump or vacuum pump. When a user's gaze is on the left square, the square is highlighted blue while the air pump is consequently actuated ON to inflate the soft robot. Conversely, when a user's gaze is fixated on the right square, the square is highlighted blue while the vacuum pump is consequently actuated ON to suck out air from the corrugated actuator. When a user's eyes are outside of both marked regions, both regions are inactivated and both vacuum and air pumps are OFF. In this manner, a soft actuator controlled by eye positioning and tracking on a computer screen has been fully implemented.

Figure 13.

(a) A Graphical User Interface showing the activation of each square region when a user looks at a(i) the left square or a (ii) the right square (b) A user demostrating the use of the soft actuator controlled via the eye tracker by wearing the soft actuator and controlling inflation and deflation cycles of the soft actuator via the Tobii Eye Tracker 4C.

Figure 13b shows a user seated in front of a computer screen with the soft robot inflating and deflating as the user looks at the left and right squares respectively. As shown the user wears the corrugated soft device via the Velcro attachment which causes flexion and extension of the forearm as eye tracking information causes the soft actuator to be deflated by the vacuum pump or inflated by the air pump. One limitation of this integrated system is due to restricted mobility. This implies that a potential user has to be restricted to a stationary position in order to use both the eye tracker and soft robot for its intended use. Also, the properties of silicone (which was used in producing the corrugated actuator) e.g. durability and structural strength changes with respect to inflation and deflation cycles, tear as a result of repeated stretching, and manual fabrication process. Aside from these limitations, the system has enormous benefit as a rehabilitation device for delivering physiotherapy exercises for forearm motions.

This integrated system might be useful for persons requiring rehabilitation in form of a physiotherapy device by providing an assistive force required to move the forearm. Individuals that have weakness of the forearm such as stroke patients often require a physiotherapist or physiotherapeutic device to enable them practice the use of the forearm so as to facilitate the ease of recovery. A physiotherapist helps these individuals gain strength in their forearm by exercising the use of these weak parts through movements of the parts. As a consequence of the soft actuator being able to provide continuous and repetitive movements through the use of eye tracking information, potential users of this system will be able to exercise the use of their forearm, on their own, controlled via the eye tracking interface, thereby acting as a physiotherapeutic device to aid rehabilitation.

5. Conclusion

The paper involved the design and implementation of a corrugated soft actuator controlled by an eye tracker. The application of this system is as a physiotherapy device for individuals that are weak in their forearm. By simply using eye movements to regulate air pressure within the actuator, these individuals can perform exercises of their forearm which will aid them in regaining the function of their forearm. Likewise, open loop control of a soft robot is possible without the use of an interfacing sensor. Because visual information from the user's eyes is utilized as feedback to check the actuator and decide whether to inflate or deflate, the application can extend to other functions such as remotely controlling a soft actuator for different movement patterns through eye tracking.

Declarations

Author contribution statement

Victoria Oguntosin: Conceived and designed the experiments; Performed the experiments; Wrote the paper.

Ademola Abdulkareem: Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.