Intelligent mobile robot for contagious disease treatments in hospitals

Abstract

This paper provides a novel and applicable work that builds a real system for disinfecting the air and surfaces of the environment in a hospital room, with a non-contact measurement system for supporting contagious disease treatments in hospitals. The system is built on an intelligent mobile robot system that operates autonomously in a simulated real treatment room. The research team uses a new positioning algorithm. It is a combination of data from the Lidar sensor, encoder, and Extended Kalman filter. The program that applies segmentation and image feature extraction algorithms is developed to meet requirements of real-time environment mapping in the room. Control algorithms for moving and avoiding obstacles are also proposed. Next, techniques for collecting health data including patient identification, body temperature, and blood oxygen index via wireless sensor network are also mentioned in the article. Analysis and experimental results show qualified outcomes and promise. The main contribution of the paper can be listed as follows.

• •Design and build a new CEE-IMR, an intelligent mobile robot that can regconize patients, guide and lead them walking in hospitals, especially keep a safe distance avoiding contagious deseases.
• •A novel framework for controlling the robot is proposed. The robot can move flexible, avoid obstacles, etc. based on advanced control algorithms. A new control mechanism is also proposed.
• •Methods of collecting data and processing medical data to support either patients or doctors to improve the effecency in hospitals in contagious disease management.

Graphical abstract

Specifications table

This table provides general information on your method.

Subject area:	Engineering
More specific subject area:	Autonomous robot (AR), control, AR assisted medical treatment, sensing systems
Name of your method:	Design an Intelligent Mobile Robot for Contagious Disease Treatments in Hospitals
Name and reference of original method:	All the reference papers are provided in the Ref. Section.
Resource availability:	All data of this work (e.g., equipment, data, software, hardware) can be shared upon request.

Background

Mobile robots provide a wide range of applications in both civil and military areas [1,2]. Mobile robots may detect, monitor, collect and analyze data, navigate among other things, depending on their intended use. Since the outbreak of the coronavirus pandemic 2019 (COVID-19) occure in 2019, new approaches for robots in a contagious world are exploited rapidly to support either hospitals or in public areas. The role of robots in the following COVID-related applications: disinfection, checking human temperature, monitoring public places, delivering food and other items, food preparation and personal interactions by distanced contacts [[3], [4], [5]]. The effective control and prevention of this epidemic remain major challenges for the international community. In addition, contagious surfaces and environments are important sources of infection that promote the spread of the virus. Therefore, environmental disinfection in healthcare facilities is important to reduce transmission, prevent hospital-acquired infections, and control disease outbreaks [6,7].

Currently, the global socio-economic development and health system reform have enabled the medical industry to move towards true "intelligence". The combination of artificial intelligence (AI) technques, 5 G and 6 G technologies, Internet of Things (IoTs), manufacturing robots, cyber-physical (CPS), and other new high technologies enrich the medical profession [8]. The advanced technologies can fully support the equipments to fulfill their tasks in hospitals or in other environments to protect human from the virus.

Prevention and control of hospital-acquired infections is a cornerstone of health safety and a core mission in improving the quality of health services. The combination of engineering technology and infection management in hospitals has led to breakthroughs in infection control technology [[9], [10], [11], [12], [13], [14]]. Healthcare organizations around the world have been optimizing hospital intelligence by gathering and organizing information from multiple perspectives to build innovative models in hospital development . Some results related to medical services avoiding contagious diseases have been shown separately due to specific missions. Robots are preferably deployed at a hospital's front desk to spread information about the hospital's various units/sections and to guide patients and visitors [9]. Nursing robots in hospital and at home for elderly care are investigated in [10]. Robots also can provide immediate medical aid after an accident is critical in order to prevent intensification of trauma [11]. Robots can help doctors far from patients to take all the physiological parameters and diagnoses a disease using audiovisual aids [12]. Delivery services inside hospitals can be carried out by mobile robots that serving food and beverages, dispensing of drugs, removing of unclean laundry, delivery of fresh bed linen, and transportation of regular and contaminated waste etc. [13]. Cleaning robots for medical surroundings appear to be capable of delivering the innovation that non-industrial robot creators predicted years ago [14]. These models help to reduce costs associated with the prevention and control of hospital sterilization and only deal with specific functions.

Recently published research works [[15], [16], [17], [18]] have solved the problems of disinfecting and cleaning the environments, but have not mentioned the problems of positioning and guiding or navigating robots that can operate correctly in disinfected environments. This is a critical issue that the robot can come into direct contact with the patient to replace the human that has not been interested by the authors.

In this work, many functions are integrated into one mobile robot, called CEE-IMR. The mobile robot shows a manuverability to move in an infected area to collect data from patients to be sent to doctors or a data processing center. The hardware and software are designed significantly to be able to adapt to hospital environments. The robot receives orders and work independently. Technical designs are different from existing work that can support effectively to either patients or doctors.

As shown in Fig. 1, we present the indirect control of blood oxygen levels (SpO2 is an important indicator to monitor and evaluate the health of COVID-19 patients in the recommended treatment regimen [19] and patient body temperature through Ther-mal Image Processing or Infrared (IR) spectral detection method [20]. Through these two parameters, the doctor does not need to have direct contact with the patient, but can still monitor and evaluate the appropriate treatment regimen.

Fig. 1.

(a) Image of robot model in the experiment, (b) Detailed mechanical drawing of the base of the robot, (c) Kinetic model of mobile robot.

The disinfection is performed by a system including spraying, mopping the floor surface, and UV-C lamp described in the design of the robot body. In addition, controlling the robot to follow the trajectory to the desired locations and some theories related to this test will be presented in the next sections. For the task of avoiding close obstacles in the local area, we have applied the VFH+ method [[21], [22], [23]] with a system of ultrasonic sensors and developed a control program that allows avoiding obstacles in the local area from 0.3 to 4metters.

The Lidar system allows the detection of features of the robot's environment to combine with encoder sensor data, which allows the robot to move precisely to the target locations. The measurement and experimental results are presented in section 3.1. With these results, it is possible to open up the possibilities of applying intelligent robots for contagious disease treatments in practice.

The rest of this work is organized as follows. The system models that combine all the functions in the robot is addressed. The proposed methods including algorithms, calculations are provided. Next, experimental results including control problems, data collecting, proccessing and analyzing are addressed. Finally, conclusions and future developments are provided.

Method details

+ A method of designing and building a new intelligent mobile robot, named CEE-IMR, that can regconize patients, guide and lead them walking in hospitals, especially keep a safe distance avoiding contagious deseases.

+ A method for controlling the robot is proposed. The robot can move flexible, avoid obstacles, etc. based on advanced control algorithms. A new control mechanism is also proposed.

+ A Method of collecting medical data and processing the data is proposed to support either patients or doctors to improve the effecency in hospitals in contagious disease management.

The proposed system models to support contagious disease treatments

Fig. 2 is a block diagram of a mobile robotic system model for contagious disease treatment support. This model has 2 main systems: The sensor system consists of turquoise blocks (Fig. 2) presented in Section 2.1; The control system consists of orange and blue blocks (Fig. 2) presented in Section 1.2.

Fig. 2.

Block diagram of CEE-IMR system.

Sensor systems

Encoder

The optical encoder has an LED light source, a light detector, a “code” disc/wheel mounted on the shaft, and output signal processor, Fig. 3. The disc has alternating opaque and transparent segments and is placed between the LED and photodetector. As the encoder shaft rotates, the light beam from the LED is interrupted by the opaque lines on the “code” disk before being picked up by the photodetector. This produces a pulse signal: light = on; no light = off. The signal is sent to the counter or controller, which will then send the signal to produce the desired function.

Fig. 3.

Structure of the rotary encoder.

In mobile robotics, the encoder is used to measure the movement (direction and speed) of each of the wheels of the robot. Determining the position of the robot by this encoder is a popular method in the world called the Odometry method.

Lidar (Light detection and ranging)

To increase the efficiency of positioning we use the Lidar sensor. Lidar is a method for determining ranges (variable distance) by targeting an object with a laser and measuring the time for the reflected light to return to the receiver. In this research, as shown in Fig. 4, the research team used the lidar method to locate the robot in the hospital using the RPLIDAR A1 sensor.

Fig. 4.

(a) RPLIDAR A1 System Composition, (b) The RPLIDAR A1 Working Schematic.

The RPLIDAR A1 is a low-cost 360° 2D laser scanner (LIDAR) solution developed by SLAMTEC. The system can perform a 360° scan within the 12 m range. The produced 2D point cloud data can be used in mapping, localization, and object/environment modelling.

Ultrasonic sensor

An ultrasonic sensor is an electronic device that measures the distance of a target object by emitting ultrasonic sound waves, and converts the reflected sound into an electrical signal. Ultrasonic waves travel faster than the speed of audible sound (the sound that humans can hear). Ultrasonic sensors have two main components: the transmitter (which emits the sound using piezoelectric crystals) and the receiver (which encounters the sound after it has travelled to and from the target).

In a hospital environment, there are many people and obstacles on the way, so the robot must be aware to avoid that obstacle. The ultrasonic sensors are used primarily as proximity sensors. They can be found in automobile self-parking technology and anticollision safety systems. Therefore, we use ultrasonic sensor SRF05 for the robot so that the robot can avoid obstacles, as illustrated in Fig. 5.

Fig. 5.

Ultrasonic sensor.

RGB camera

The robot is designed in addition to the functions mentioned above, it also has the function of guiding the patient to move in the hospital. Accordingly, the robot both moves and tracks the patient's face to maintain a distance from the patient during movement. If the patient stops or slows down, the robot will stop to wait, which is made possible by a method that predicts the distance from the image sensor to the person's face using the monocular camera. We use the Logitech BRIO 4 K Camera, as shown in Fig. 6, installed on the robot to capture the patient's face image.

Fig. 6.

Logitech BRIO 4 K.

Infrared thermal image sensor and SpO2 sensor

The temperature sensor used in this research is Panasonic's AMG8833, as shown in Fig. 7. An infrared thermal image sensor with 64 thermal pixels distributed into a 2D 8 × 8 matrix, can measure the temperature of an object in the range of from 0 °C to 80 °C. Non-contact infrared sensor with measuring distance up to 7 m under ideal conditions. Sensor measuring angle up to 60° in both horizontal and vertical. The sampling rate is 10 fps (frame per second).

Fig. 7.

(a) Infrared thermal image sensor – AMG8833 sensor Node, (b) Case for the AMG8833 sensor Node.

The robot is set to move at a speed of 0,2 m/s to 0,6 m/s. Therefore, under ideal conditions, it is possible to collect temperature data even while the robot is moving.

The SpO2 sensor, as shown in Fig. 8, is the MAX30102 MH-ET LIVE module that integrates the MAX30102 heart rate and SpO2 sensor. The MAX30102 sensor contains two light-emitting diodes (LEDs), one infrared (peak wavelength 880 nm) and the other red (peak wavelength 660 nm); along with a photodiode that is specific for wavelengths between 600 and 900 nm. The values measured by the sensor are transmitted with a checksum, such that the validity of the data is verified when it is received by the supporting microcontroller (3). According to Fig. 8, this sensor is worn on the patient's finger.

Fig. 8.

(a) Infrared thermal image sensor – MAX30102 sensor Node, (b) Case for the MAX30102 sensor Node.

The control system

The sensory information network of the robot is shown in Fig. 2, in which the computer is the central control unit. Due to the characteristics of different information rates between sensors (the information here is point-to-multipoint), a special design for the communication network in the robot has been implemented. As shown in Fig. 2, the ultrasonic sensors, RPlidar and robot actuators (motors) are connected to the robot's control unit (serial communication RS-232C); The RGB camera sensor is connected to the control unit via a high-speed communication channel; Other path (Modules Relay, UV germicidal irradiation, disinfectant spray pump). These blocks connect to the control board (via RS-485 communication standard) with a point-to-multipoint line according to the Modbus/RTU polling protocol.

The control program for the system to collect patient data and disinfect the floor during robot movement can be summarized as follows:

Starting from the hardware configuration shown above, the software that collects information from the sensors can be divided into modules with the following specific tasks:

• -The program for collecting and processing data from the sensor RPLidar is a special type developed specifically with the C++ language;
• -Camera image information is collected directly into the computer and processed through an image processing program, which is developed in Visual C ++ environment with Intel's OpenCV tool;
• -Control program for two optical encoder measuring circuits (mounted on motor shafts) connected directly to a microcomputer (via RS-232C);
• -The remaining information is transmitted and received between 14 network nodes and control nodes from the computer (2 nodes for channels Relay module, 12 node for 12 ultrasonic sensors) via transmission line RS-485 using Modbus/RTU protocol;

Fig. 9 shows an overview of the navigation process for a mobile robot, which can be summarized as follows.

• 1.Robot collects data through sensors;
• 2.Determine the position of the robot;
• 3.Mapping (if necessary or import existing global maps);
• 4.plan the way;
• 5.Control the moving robot to follow the trajectory to the destination and avoid obstacles (if any).

Fig. 9.

Flowchart of navigation process for mobile robot.

Collection of patient health data for medical treatment

As mentioned above, SpO2 index and body temperature are two important measurements in monitoring and treating covid 19 patients. Our robot will collect these two kinds of data automatically. Data collection is done automatically. The robot moves close to the patient's bed within a radius of 1 meter, the sensor nodes will automatically connect to the Wi-Fi network emitted by a router placed on the robot. And those nodes actively send measured data to the central computer according to the MQTT protocol [24]. A typical MQTT is a TCP-based protocol uses a topic-based publish-subscribe architecture, as illustrated in Fig. 10. It's used to establish communication between multiple devices. This communication protocol is suitable for transferring data between resource-constrained devices with low bandwidth and low power requirements. Therefore, this messaging protocol is widely used for communication in IoT Framework.

Fig. 10.

Message exchange in the MQTT protocol.

As shown in Fig. 11, there are, 1- the publishers, which are IoT devices, usually sensors that share data; 2- the subscribers, which are IoT devices that receive shared data by the publishers; 3- the brokers, which are entities that manage topics and message exchange, and can be implemented within any device in the network, including the publishers and subscribers. When new data is generated, publishers send messages to the broker, in order to update a given topic. Every time a topic is updated, the broker broadcasts the new data to every subscriber of that topic. Fig. 11 represents a sequence diagram regarding the message exchange in the MQTT protocol. With the above technical characteristics, MQTT can work in unstable transmission conditions. Hence it is suitable for our robot model.

Fig. 11.

Diagram of sensor data exchange using MQTT protocol.

In Fig. 11, the wireless sensor data collection system is set up with 3 clients in which AMG8833 and MAX30102 are 2 sensor nodes (called MQTT clients) that collect data independently of each other and with the robot. Every time the sensor nodes "find" the Wi-Fi network emitted by the robot, they connect to a server (which is an open-source Mosquitto broker application that runs in the background inside the robot's computer-https://mosquitto.org/). All collected sensor data is stored in the software on the computer to provide medical treatment data to the doctor.

The proposed methods

Program to merge data of sensors for mobile robot positioning

Positioning for robots using extended Kalman filters

As shown in Fig. 1(c) the coordinate system and parameter symbols of the mobile robot are designed to move on the flat floor in the room, where (X_G,Y_G) are the axes of the global coordinate system of the room, (X_R,Y_R) are local coordinate system axes that match the robot's center. Let the angular velocities of the right and left wheels be ω_R and ω_L respectively, and the sampling interval Δt is sufficiently short. So that the input control signals that are the displacement increments applied to these wheels are:

$$\begin{array}{ccc}& & \Delta s_R=\Delta tR{\omega }_R\hfill \\ & & \Delta s_L=\Delta tR{\omega }_L\hfill \end{array}$$ (1)

From here, the distance increment of the robot center Δs is:

$$\Delta s=\frac{\Delta s_R+\Delta s_L}{2}$$ (2)

And the angular increment in which the robot rotates Δθ is:

$$\Delta \theta =\frac{\Delta s_R-\Delta s_L}{L}$$ (3)

So the equation of state of the robot at time k in the global coordinate system updated from time k–1 is as follows:

$$\left[\begin{array}{c}{x}_k\\ y_k\\ {\theta }_k\end{array}\right]=\left[\begin{array}{c}{x}_{k-1}\\ y_{k-1}\\ {\theta }_{k-1}\end{array}\right]+\left[\begin{array}{c}\Delta s_{k}cos\left({\theta }_{k-1}+\Delta {\theta }_k/2\right)\\ \Delta s_{k}sin\left({\theta }_{k-1}+\Delta {\theta }_k/2\right)\\ \Delta {\theta }_k\end{array}\right]$$ (4)

The displacement increments Δs_R and Δs_L (proportional to the set speeds of the 2 wheels ω_R and ω_L as measured by the crankshaft encoder sensors) are input variables affected by errors systematic and random errors.

In practice, the system (4) encounters systematic errors (such as dimensional errors of mechanical parts, shaft offsets, encoder resolution limits, …) and non-systematic errors. In this paper, the errors are proportional to the two-wheel displacement increments Δs_R and Δs_L, these increments when subjected to noise can be expressed as the sum of the two components including the value identification and interference:

$$\begin{array}{ccc}& & \Delta s_R=\Delta s_{R0}+{\epsilon }_R\hfill \\ & & \Delta s_L=\Delta s_{L0}+{\epsilon }_L\hfill \end{array}$$ (5)

where $\Delta s_{R0}$ and $\Delta s_{L0}$ are the nominal values of the input signal; ${\epsilon }_R$ and ${\epsilon }_L$ are the values of the white noise process, which are independent, have zero mean and are normally distributed.

The equation of state (4) can be written in more detail as follows:

$$\left[\begin{array}{c}{x}_k\\ y_k\\ {\theta }_k\end{array}\right]=\left[\begin{array}{c}{x}_{k-1}\\ y_{k-1}\\ {\theta }_{k-1}\end{array}\right]+\left[\begin{array}{c}\frac{\left(\Delta s_{R0\left(k\right)}+{\epsilon }_{R\left(k\right)}\right)+\left(\Delta s_{L0\left(k\right)}+{\epsilon }_{L\left(k\right)}\right)}{2}\cos \left({\theta }_{k-1}+\frac{\left(\Delta s_{R0\left(k\right)}+{\epsilon }_{R\left(k\right)}\right)-\left(\Delta s_{L0\left(k\right)}+{\epsilon }_{L\left(k\right)}\right)}{2L}\right)\\ \frac{\left(\Delta s_{R0\left(k\right)}+{\epsilon }_{R\left(k\right)}\right)+\left(\Delta s_{L0\left(k\right)}+{\epsilon }_{L\left(k\right)}\right)}{2}\sin \left({\theta }_{k-1}+\frac{\left(\Delta s_{R0\left(k\right)}+{\epsilon }_{R\left(k\right)}\right)-\left(\Delta s_{L0\left(k\right)}+{\epsilon }_{L\left(k\right)}\right)}{2L}\right)\\ \frac{\left(\Delta s_{R0\left(k\right)}+{\epsilon }_{R\left(k\right)}\right)-\left(\Delta s_{L0\left(k\right)}+{\epsilon }_{L\left(k\right)}\right)}{L}\end{array}\right]$$ (6)

Due to the cumulative nature of systematic errors, the accuracy of the position estimation will be worse as the robot moves. Many studies have tried to increase the reliability of this positioning, including the use of the Kalman filter. We also applied sensor fusion using the Kalman filtering algorithm for positioning to the robot and obtained results that improved the quality of these measurements significantly [25,26]. It can be summarized as follows:

If the state vector $\mathbf{x}\in {\Re }^{\mathrm{n}}$ of a controlled process is represented by Eq. (12). This nonlinear equation has the form:

$${\mathit{x}}_k=f\left({\mathit{x}}_{k-1},{\mathit{u}}_{k-1},{\mathit{w}}_{k-1}\right)$$ (7)

where f is a system function with $\mathit{x}={\left[xy\theta \right]}^T$ is the state vector, $\mathit{u}={\left[\Delta s_R\Delta s_L\right]}^T$ is the input control vector and $\mathit{w}=\left[{\epsilon }_R{\epsilon }_L\right]$ is the process noise.

This state is observed by several measurements that make up the output vector z:

$${\mathit{z}}_k=h\left({\mathit{x}}_k,{\mathit{v}}_k\right)$$ (8)

where h is the measurement function and $\mathit{v}=\left[{\epsilon }_R{\epsilon }_L\right]$ is the measurement noise affecting the readings of displacement increments from the rotary axis encoders.

w_k and v_k are independent, are white noise, have a normal probability distribution with covariance matrices Q and R respectively:

$$P\left({\mathit{w}}_k\right)\sim \mathit{N}\left(0,{\mathit{Q}}_k\right);P\left({\mathit{v}}_k\right)\sim \mathit{N}\left(0,{\mathit{R}}_k\right);E\left({\mathit{w}}_i{{\mathit{v}}_j}^T\right)=0$$ (9)

Define matrices in sensor fusion steps

The speed and distance of the robot are shown in Fig. 1(c) is measured by the Odometry method based on the function f in the system Eq. (7).

• •Starting phase: From step k = 0, the robot starts from a known initial position with a posterior state estimate ${\widehat{\mathbf{x}}}_0$ and the covariance of the error estimate P₀.
• •
  Prediction phase: At each time step k, the filter propagates the state x and covariance P of the system in the previous step to the current step using the time update equation:

  $${\widehat{\mathit{x}}}_k^{-}=f\left({\widehat{\mathit{x}}}_{k-1},{\mathit{u}}_{k-1},\mathit{0}\right)$$ (10)

  $${\mathit{P}}_k^{-}={\mathit{A}}_k{\mathit{P}}_{k-1}{\mathit{A}}_k^T+{\mathit{W}}_k{\mathit{Q}}_{k-1}{\mathit{W}}_k^T$$ (11)

Here, input control signals $\mathit{u}={\left[\Delta s_R\Delta s_L\right]}^T$ measured by the encoder sensor.

To get the filter's matrix P, the matrices Q and A are defined as follows:

• ○
  Q_k is the input noise covariance matrix with size [2 × 2], we have:

  $${\mathit{Q}}_k=covar\left(\Delta s_{R\left(k\right)},\Delta s_{L\left(k\right)}\right)=\left[\begin{array}{cc}{\delta }_R\left|\Delta s_{R\left(k\right)}\right|& 0\\ 0& {\delta }_L\left|\Delta s_{L\left(k\right)}\right|\end{array}\right]$$ (12)

Here, δ_R and δ_L are error constants representing the uncertainty parameters of the motor control circuit and the interaction of the wheel with the floor, and they are independent of each other. Thus, the variance of the process noise is proportional to the absolute value of the increments distance travelled by each wheel Δs_R and Δs_L. These scale constants depend on the structure of the robot and the environment. They are determined experimentally as ${\delta }_R={\delta }_L\equiv \delta =0,0003.$

• ○
  A_k is a matrix of size [3 × 3], calculated by:

  $${\mathit{A}}_k=\left[\begin{array}{ccc}{\displaystyle \frac{\partial f_x}{\partial x}}& {\displaystyle \frac{\partial f_x}{\partial y}}& {\displaystyle \frac{\partial f_x}{\partial \theta }}\\ {\displaystyle \frac{\partial f_y}{\partial x}}& {\displaystyle \frac{\partial f_y}{\partial y}}& {\displaystyle \frac{\partial f_y}{\partial \theta }}\\ {\displaystyle \frac{\partial f_{\theta }}{\partial x}}& {\displaystyle \frac{\partial f_{\theta }}{\partial y}}& {\displaystyle \frac{\partial f_{\theta }}{\partial \theta }}\end{array}\right]=\left[\begin{array}{ccc}1& 0& -\Delta s_k\sin \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)\\ 0& 1& \Delta s_k\cos \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)\\ 0& 0& 1\end{array}\right]$$ (13)
• ○
  W_k is a matrix of size [3 × 2], calculated by:

  $${\mathit{W}}_k=\left[\begin{array}{cc}{\displaystyle \frac{1}{2}}\cos \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)+{\displaystyle \frac{\Delta s_k}{2L}}\sin \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)& {\displaystyle \frac{1}{2}}\cos \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)+{\displaystyle \frac{\Delta s_k}{2L}}\sin \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)\\ {\displaystyle \frac{1}{2}}\sin \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)+{\displaystyle \frac{\Delta s_k}{2L}}\cos \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)& {\displaystyle \frac{1}{2}}\cos \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)-{\displaystyle \frac{\Delta s_k}{2L}}\sin \left({\theta }_{k-1}+{\displaystyle \frac{\Delta {\theta }_k}{2}}\right)\\ {\displaystyle \frac{1}{L}}& -{\displaystyle \frac{1}{L}}\end{array}\right]$$ (14)

• •
  Correction phase: The filter corrects the a priori state estimate with z_k measurements by the following measure update equations:

  $${\mathit{K}}_k={\mathit{P}}_k^{-}{\mathit{H}}_k^T{({\mathit{H}}_k{\mathit{P}}_k^{-}{\mathit{H}}_k^T+{\mathit{R}}_k)}^{-1}$$ (15)

  $${\widehat{\mathit{x}}}_k={\widehat{\mathit{x}}}_k^{-}+{\mathit{K}}_k\left({\mathit{z}}_k-\mathit{h}\left({\widehat{\mathit{x}}}_k^{-}\right)\right)$$ (16)

  $${\mathit{P}}_k=\left(\mathit{I}-{\mathit{K}}_k{\mathit{H}}_k\right){\mathit{P}}_k^{-}$$ (17)

In this case we use an absolute measurement using the RPLIDAR sensor. This sensor allows the detection of features of the environment around the robot, which are lines detected in the environment (for example, a line that cuts across a flat wall in a room).

By a transformation such as the Hough transform, we have two parameters of distance r from the line to the origin and angle ψ between the line as shown in Fig. 12. The global map of the built environment consists of a set of Line segments are described by the parameters β_j and ρ_j. The equation of the straight line in canonical form is:

$$x_G\cos {\beta }_j+y_G\sin {\beta }_j={\rho }_j$$ (18)

Fig. 12.

Lidar sensor measures 2 parameters of a straight line in the environment.

As the robot moves, a local map of the environment is built with coordinates mounted on the robot. It also includes a set of line segments described by the equation:

$$x_R\cos {\psi }_i+y_R\sin {\psi }_i=r_i$$ (19)

where ψ_i and r_i are the parameters of the i^th line among N.

Line segments of the local and global environment will be detected as matches. Line segments that are matched to the local map will be collected into vector z_k, used as input for the EKF editing step:

$${\mathit{z}}_k={[r_{1k},{\psi }_{1k},...,r_{Nk},{\psi }_{Nk}]}^T$$ (20)

Calculate the matrices R, h and V

From the coordinates and direction of the robot estimated by the odometry method, the parameters β_j and ρ_j of the j^th line segment in the global map are converted to parameters r_i and ψ_i according to the local coordinate system of the robot. They are calculated by:

$$\begin{array}{ccc}& & \left[\begin{array}{c}{r}_i\\ {\psi }_i\end{array}\right]=\left[\begin{array}{c}\left|C_j\right|\\ {\beta }_j-\theta +\pi \left(-0,5sign\left(C_j\right)+0,5\right)\end{array}\right];\hfill \\ & & C_j={\rho }_j-x\cos {\beta }_j-y\sin {\beta }_j\hfill \end{array}$$ (21)

ψ_i and r_i are the measurements that are directly affected by the measurement noises of the RPLIDAR sensor. Call these measurement disturbances ε_r and ε_ψ, then the measured component vector from the laser sensor when including the interferences will be:

$$\left[\begin{array}{c}{r}_i\\ {\psi }_i\end{array}\right]=\left[\begin{array}{c}\left|C_j\right|\\ {\beta }_j-{\theta }_r+\left(-0,5sign\left(C_j\right)+0,5\right)\pi \end{array}\right]+\left[\begin{array}{c}{\epsilon }_r\\ {\epsilon }_{\psi }\end{array}\right]$$ (22)

Thus, when adding a Lidar sensor, the measurement function h is added 2 N components r_i and ψ_i. When k is hidden, we have the following matrices:

• -
  The matrix H_odl has size [(2 N) × 3] and is equal to:

  $$H_{odl}=\left[\begin{array}{ccc}{\displaystyle \frac{\partial r_1}{\partial x}}& {\displaystyle \frac{\partial r_1}{\partial y}}& {\displaystyle \frac{\partial r_1}{\partial \theta }}\\ {\displaystyle \frac{\partial {\psi }_1}{\partial x}}& {\displaystyle \frac{\partial {\psi }_1}{\partial y}}& {\displaystyle \frac{\partial {\psi }_1}{\partial \theta }}\\ ⋮& ⋮& ⋮\\ {\displaystyle \frac{\partial r_N}{\partial x}}& {\displaystyle \frac{\partial r_N}{\partial y}}& {\displaystyle \frac{\partial r_N}{\partial \theta }}\\ {\displaystyle \frac{\partial {\psi }_N}{\partial x}}& {\displaystyle \frac{\partial {\psi }_N}{\partial y}}& {\displaystyle \frac{\partial {\psi }_N}{\partial \theta }}\end{array}\right]=\left[\begin{array}{ccc}-c_1\cos {\beta }_1& -c_1\sin {\beta }_1& 0\\ 0& 0& -1\\ ⋮& ⋮& ⋮\\ -c_1\cos {\beta }_N& -c_1\cos {\beta }_N& 0\\ 0& 0& -1\end{array}\right]$$ (23)

where c_i is the sign function sign(C_i).

• -
  The matrix V_odl has size [(2 N) × 3] and is equal to:

  $$V_{odl}=\left[\begin{array}{ccc}{\displaystyle \frac{\partial r_1}{\partial {\epsilon }_{\phi }}}& {\displaystyle \frac{\partial r_1}{\partial {\epsilon }_r}}& {\displaystyle \frac{\partial r_1}{\partial {\epsilon }_{\psi }}}\\ {\displaystyle \frac{\partial {\psi }_1}{\partial {\epsilon }_{\phi }}}& {\displaystyle \frac{\partial {\psi }_1}{\partial {\epsilon }_r}}& {\displaystyle \frac{\partial {\psi }_1}{\partial {\epsilon }_{\psi }}}\\ ⋮& ⋮& ⋮\\ {\displaystyle \frac{\partial r_N}{\partial {\epsilon }_{\phi }}}& {\displaystyle \frac{\partial r_N}{\partial {\epsilon }_r}}& {\displaystyle \frac{\partial r_N}{\partial {\epsilon }_{\psi }}}\\ {\displaystyle \frac{\partial {\psi }_N}{\partial {\epsilon }_{\phi }}}& {\displaystyle \frac{\partial {\psi }_N}{\partial {\epsilon }_r}}& {\displaystyle \frac{\partial {\psi }_N}{\partial {\epsilon }_{\psi }}}\end{array}\right]=\left[\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\\ ⋮& ⋮& ⋮\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$ (24)
• -
  The matrix R_odl has size [(2 N) × (2 N)] and is equal to:

  $$R_{odl}=\left[\begin{array}{cccccc}var\left(r_1\right)& 0& 0& \cdots & 0& 0\\ 0& var\left({\psi }_1\right)& 0& \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & 0& var\left(r_N\right)& 0\\ 0& 0& \cdots & 0& 0& var\left({\psi }_N\right)\end{array}\right]$$ (25)

Identification of Kalman filter's parameters from the Lidar

This section describes the algorithm to extract line segments from a set of points based on the least square quadratic (LSQ) method. The point cloud was obtained from a scanner with an 360° field of view and a 1° angular resolution. The data collected by the RPLIDAR was first converted into Cartesian coordination and stored in an array according to the equation:

$$x=r\cos \left(\phi {\displaystyle \frac{\pi }{180}}\right);y=r\sin \left(\phi {\displaystyle \frac{\pi }{180}}\right)$$ (26)

where r and φ are the range and bearing from robot position to obstacles respectively.

Points that lie too close to robot position (<40 cm) were treated as noise and were eliminated. Because the RPLIDAR scans from right to left, neighboring points have high chance to lie on a same landmark such as on the wall. The LSQ was applied to a group of N₁ consecutive laser readings starting from the first points to find the best fit line of these points. A distance threshold disT was predefined and compared with the distance from each point in the group to the best fit line. If there were more than N₂ of the points in the group (in our research, N₂ = 0,75N₁), the distance of which is fewer than the disT, a line was detected and a new best fit line was calculated based on these points. The point with the smallest index would be saved as one end of the line segment. The slope m and intercept k of the line were computed using the formula:

$$m={\displaystyle \frac{\sum xy-{\displaystyle \frac{\left(\sum x\right)\left(\sum y\right)}{n}}}{\sum x^2-{\displaystyle \frac{{(\sum x)}^2}{n}}}};k=\overline{y}-m\overline{x}$$ (27)

where n is the total number of data points, $\overline{x}$ and $\overline{y}$ are the mean of the x- and y- coordinates of the data points respectively. The algorithm would then be used to find the remaining points of the cloud to add more points to the line segment. Points that lie too far from their previous neighbors in the array would be ignored. The last point that met the distance threshold was then treated as the other end of the found line segment. The algorithm was repeated with the new group until all elements of the collected point cloud were checked.

Nevertheless, a fixed threshold disT did not work for all scans. Experiment results show that in about 10% of the scans, the threshold of the distance from one point to the best fit line should be larger in order to detect lines from data points. However, if the threshold applied for all scans was too large, extracted line segments would be not smooth and could affect the matching result. In this paper, a simple dynamic threshold was developed to solve this problem. A maximal threshold disT_max was defined as the largest distance to determine if a point belongs to a line. For each group of points, instead of comparing the distance from each point to the best fit line with fixed disT, all distances would be stored in an array and sorted in an ascending order. If the N₂th distance was smaller than the maximal threshold disT_max, it would be chosen as the distance threshold for this line segment. Therefore, the number of distance threshold in each scan would be the same as the number of detected line segments.

Local and global maps matching

The extracted line segments of same landmarks from local and global maps are then matched together using a straight-forward algorithm. The extracted line segments from global map G_j and local map L_j are described as follows:

$$\begin{array}{ccc}\hfill G_j& =& \left[x_{1G,j}{x}_{2G,j}{y}_{1G,j}{x}_{2G,j}\right]j=1\dots n_G\hfill \\ \hfill L_i& =& \left[x_{1L,i}{x}_{2L,i}{y}_{1L,i}{y}_{2L,i}\right]i=1\dots n_L\hfill \end{array}$$ (28)

where (x₁,y₁) and (x₂,y₂) are Cartesian coordinate of two end points of extracted line segments respectively, n_G and n_L are number of extracted global and local line segments.

The detected global line segments are first transformed into local map with the equation:

$$G_{Tj}=\mathit{G}\left(\left[\begin{array}{cc}{x}_{1G,j}& y_{1G,j}\\ x_{2G,j}& y_{2G,j}\end{array}\right]-\left[\begin{array}{cc}{x}_0& y_0\\ x_0& y_0\end{array}\right]\right)$$ (29)

where $\mathit{G}=\left(\begin{array}{cc}\cos {\phi }_0& -\sin {\phi }_0\\ \sin {\phi }_0& cos{\phi }_0\end{array}\right)$ is the conversion matrix, ${\left[x_0{y}_0{\theta }_0\right]}^T$ are indicators of robot's position in global map estimated by Odometry. Each local line segment L_i is compared with all transformed global line segments G_T,j, and two line segments are considered as “matched” if their range and bearing to robot position are approximately the same and the overlapping rate between them is less than previous defined threshold [27].

The range and bearing of global and local line segments to robot position in robot's coordinate are presented as (ρ_G,j,ψ_G,j) and (ρ_,Li,ψ_L,i), respectively as shown in Fig. 13.

Fig. 13.

The range and bearing of global and local line segments to robot's position in robot's coordinate.

The overlapping rate between the local line segment L_i and the transformed global line segment G_T,j is defined as follows:

$$\begin{array}{ccc}& & O_k\left(a_k,b_k\right)=\left|a_k+b_k-{\overline{G}}_{T,j}\right|\hfill \\ & & O_k\left(c_k,d_k\right)=\left|c_k+d_k-{\overline{G}}_{T,j}\right|\hfill \\ & & k=\left(j-1\right)n_L+i\hfill \\ & & k=1,2,...,n_L{n}_G-1,n_L{n}_{G}k=1,2,...,n_L{n}_G-1,n_L{n}_G\hfill \end{array}$$ (30)

where G_T,j is the length of transformed global line segment; a_k, b_k, c_k and d_k are Euclidean distances between the end points of the line segments Li and G_T,j as shown in Fig. 14.

Fig. 14.

The overlapping parameters between global and local line segments.

The inequations below represent conditions for matching local and transformed global line segments in robot's coordinate. Two line segments L_i and G_T,j are matched if all following conditions are met:

$$\{\begin{array}{c}{O}_k\left(a_k,b_k\right)<T\hfill \\ O_k\left(c_k,d_k\right)<T\hfill \\ {({\rho }_{L,i}-{\rho }_{G,j})}^2<T_{\rho }\hfill \\ {({\psi }_{L,i}-{\psi }_{G,j})}^2<T_{\psi }\hfill \end{array}$$ (31)

where T, T_ρ, T_ψ, are predefined threshold.

Estimation of line parameter's covariances

In order to compute the measurement covariance for the EKF, each extracted line segment could be presented as (ρ, ψ) parameters where ρ stands for the perpendicular distance from robot's position to the line and ψ is the line orientation:

$$\rho ={\displaystyle \frac{\left|-k\right|}{\sqrt{m^2+1}}};\psi =\arctan 2\left({\displaystyle \frac{k}{1+m^2}},{\displaystyle \frac{-mk}{1+m^2}}\right)$$ (32)

where (m,k) are slope and intercept of detected line which were computed in (33).

From the solution give by Derich [28], the measurement covariance matrix, R, can be calculated with the assumption that each data point has the same Cartesian uncertainty:

$$\begin{array}{ccc}& & R_i={\displaystyle \frac{a_i{\sigma }_{yy}^2-b_i{\sigma }_{xy}^2+c_i{\sigma }_{xx}^2}{{(a_i-c_i)}^2+{b_i}^2}}\left(\begin{array}{cc}1& -e\\ -e& e^2\end{array}\right)+\left(\begin{array}{cc}0& 0\\ 0& {\displaystyle \frac{{\sigma }_{yy}^2{\cos }^2{\phi }_i+{\sigma }_{xx}^2{\sin }^2{\phi }_i-2{\sigma }_{xy}^2\sin {\phi }_i\cos {\phi }_i}{n}}\end{array}\right)\cong \left[\begin{array}{cc}v\text{ar}\left(r_i\right)& 0\\ 0& v\text{ar}\left({\psi }_i\right)\end{array}\right]\hfill \\ & & \text{where}:e_i=\overline{y}\cos {\psi }_i-\overline{x}\sin {\psi }_i;{\phi }_i=\left({\psi }_i+{\displaystyle \frac{\pi }{\overline{x}}}\right);\overline{x}={\displaystyle \frac{1}{n}}{x}_j;\overline{y}={\displaystyle \frac{1}{n}}{y}_j;a_i=\sum {(x_j-\overline{x})}^2;b_i=2\sum \left(x_j-\overline{x}\right)\left(y_j-\overline{y}\right);\hfill \\ & & c_i=\sum {(y_j-\overline{y})}^2\hfill \end{array}$$ (33)

Control the robot to track the trajectory

The main goal is to control the mobile robot to track a certain trajectory. A different trajectory with a path with time constraints added to it, which makes the control target not only minimize the distance between the robot and the path, but also to ensure the travel time. We define the actual robot state as: $X={\left[xy\theta \right]}^T$ and according to the pattern trajectory is: $X_{\mathrm{r}}={\left[{\mathrm{x}}_{\mathrm{r}}{\mathrm{y}}_{\mathrm{r}}{\theta }_{\mathrm{r}}\right]}^{\mathrm{T}}$

When the robot moves, the error will appear:

$$e=\left[\begin{array}{c}{e}_1\\ e_2\\ e_3\end{array}\right]=\left[\begin{array}{ccc}\cos \theta & \sin \theta & 0\\ -\sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}_r-x\\ y_r-y\\ {\theta }_r-\theta \end{array}\right]$$ (34)

From the kinetic and derivative model (40), we get the error model as follows:

$$\left[\begin{array}{c}{\dot{e}}_1\\ {\dot{e}}_2\\ {\dot{e}}_3\end{array}\right]=\left[\begin{array}{cc}\cos e_3& 0\\ \sin e_3& 0\\ 0& 1\end{array}\right]\left[\begin{array}{c}{v}_r\\ {\omega }_r\end{array}\right]+\left[\begin{array}{cc}-1& e_2\\ 0& -e_1\\ 0& -1\end{array}\right]\left[\begin{array}{c}v\\ \omega \end{array}\right]$$ (35)

where v_r, ω_r are the linear and angular velocity of the robot according to the trajectory.

The controller for the robot is built as follows:

$$\left[\begin{array}{c}v\\ \omega \end{array}\right]=\left[\begin{array}{c}{v}_r\cos e_3\\ {\omega }_r\end{array}\right]+\left[\begin{array}{c}{v}_{fb}\\ {\omega }_{fb}\end{array}\right]$$ (36)

where v_fb, ω_fb are the feedback signal of the controller, are selected as follows:

$$\begin{array}{ccc}& & v_{fb}=k_1{e}_1\hfill \\ & & {\omega }_{fb}=k_2{v}_r{\displaystyle \frac{\sin e_3}{e_3}}{e}_2+k_3{e}_3\hfill \end{array}$$ (37)

where k₁>0, k₂ >0 and k₃ are the coefficients.

Then the control law for tracing the trajectory will be rewritten as:

$$\left[\begin{array}{c}v\\ \omega \end{array}\right]=\left[\begin{array}{c}{v}_r\cos e_3+k_1{e}_1\\ {\omega }_r+k_2{v}_r{\displaystyle \frac{\sin e_3}{e_3}}{e}_2+k_3{e}_3\end{array}\right]$$ (38)

Controlling the robot to avoid obstacles with VFH+ using ultrasonic sensors

Along the way, the robot must be able to detect and avoid unexpected obstacles. For these cases, a proximity sensor system is used. That is 12 ultrasonic distance sensors installed on the robot as shown in Fig. 15(a), allowing to detect obstacles in front and two sides of the robot.

Fig. 15.

(a). Layout diagram of ultrasonic sensors; (b) Update the Histogram Grid by continuously sampling the ultrasonic sensors while moving.

The VFH+ method uses a histogram grid to map the environment around the robot. This map is continuously updated with distance to obstacle data obtained from the ultrasonic sensors mounted on the robot as shown in Fig. 15(b). The method will find the optimal direction of movement when encountering obstacles, and appropriate velocity control for the robot (linear velocity, angular velocity).

Histogram grid

In this step, a two-dimensional Cartesian histogram grid (C) is generated containing the information transmitted from the ultrasonic sensors (the selected C has dimensions of 81 × 81 and a resolution of 0.1 m/cell). Each grid cell C[i, j] contains a value c_i,j representing the reliability of the existence of an obstacle at the coordinate position (i, j). The grid is filled using distances measured by ultrasonic sensors, each of which updates the value of only one cell at a time.

Polar histogram

In the second step, a window C∗ has a fixed orientation, whose center is attached to the robot and moves along its motion (the selected C∗ is 33 × 33). It is called the active window and it overlays the histogram grid C at the robot's current position. The content of each active cell in the map grid is treated as an obstacle vector, its direction β being towards the center of the active region (robot center point (RCP)):

$${\beta }_{i,j}={\tan }^{-1}\left({\displaystyle \frac{y_i-y_o}{x_i-x_0}}\right)$$ (39)

The vector magnitude of an active cell C[i, j] is given by:

$$m_{i,j}=c_{i,j}^2\left(a-b{d}_{i,j}^2\right)$$ (40)

where a and b are positive constants, c_i,j is the certainty value of an active cell (i, j), d_{i, j} distance from each cell to the center of the active area (RCP), m_{i, j} is the magnitude of the obstacle vector at the cell (i, j), (x₀, y₀) is the coordinate of the robot center, (x_i, y_i) is the coordinate of each cell (i, j).

Based on the obstacle vectors, the Primary Polar Histogram H^p is built. H^p has an arbitrary angular resolution α so that n = 360°/α is an integer. Each angular sector k corresponds to a discrete angle kα.

For each obstacle cell, the enlargement angle γ_{i, j} is defined by:

$${\gamma }_{i,j}=\text{arcsin}{\displaystyle \frac{r}{d_{i,j}}}$$ (41)

where r is the radius of the robot's safe zone.

For each sector k, the polar obstacle density is then calculated by:

$$H_k^p=\sum m_{i,j}{h}_{i,j}^{\prime }$$ (42)

with:

$$h_{i,j}^{\prime }=\{\begin{array}{cc}1& ifk\alpha \in \left[{\beta }_{i,j}-{\gamma }_{i,j},{\beta }_{i,j}+{\gamma }_{i,j}\right]\\ 0& otherwise\end{array}$$ (43)

From the Polar Histogram we can find out which direction is free so that the robot can move in that direction without encountering obstacles. The polar chart is updated according to the following rules:

$$H_{k,i}^b=\{\begin{array}{c}1if{H}_k^p>{\tau }_{high}\hfill \\ 0if{H}_k^p<{\tau }_{low}\hfill \\ H_{k,i-1}^{b}otherwise\hfill \end{array}$$ (44)

where τ_high and τ_low are high thresholds and low thresholds are chosen to ensure that obstacles too far away from the robot do not affect the robot's movement, even though the obstacle is in front of the moving direction.

Patient identification

To guide the patient by maintaining the distance between the robot and the patient during the robot movement, we apply the face detection technique [29]. From there, determine the height of the face in the image and calculate the distance from the image sensor to the face of the person standing in front of the camera, applying the principle of similarity of 2 triangles as shown in Fig. 16, Fig. 17.

Fig. 16.

The method of determining the focal length of the camera.

Fig. 17.

Determine the distance using the triangle similarity method.

Determine the distance d from the formula $a/f=\text{tan}{\theta }_1=h/d$. We can calculate the distance of d as:

$$d=h\times {\displaystyle \frac{f}{a}}$$ (45)

where a is the height of the human face in the image, f is the focal length, h is the height of the actual human face, d is the distance from the person to the camera.

Disinfection and sterilization

The sterilization and disinfection components of the robot are described in Fig. 1(a) and 1(b). In which the UV lamp is equipped at the back for the purpose of sterilization. UV lamp with a power of 36 W, wavelength 253.7 nm, the glass shell of the lamp has a filter function of 185 nm wavelength to create ozone in the air.

Disinfection is carried out as follows: in front of the robot body, there is a container of disinfectant solution with a capacity of 20l Disinfectant solution containing 0.1% active Chlorine (as recommended [30]. And will be evenly sprayed on the surface of the disinfected special treatment room with a dosage of 0.3 – 0.5l/m² by two nozzles that have been arranged under the front chassis of the robot. With the equipment arrangement as shown in Fig. 1(b), the disinfectant solution will be sprayed forward and diffused lower than the patient's lying position as recommended by medical authorities. With 1 meter of movement of the robot, the amount of sprayed solution will be equivalent to 1 ml (at each nozzle). Thus, with the carrying of 20l of the solution, the robot will disinfect an entire room with a charge of 60m² over 150 times.

In addition to disinfection, the robot is also specially equipped with a floor cleaning function according to the robot's movement diagram. Below the robot is arranged 4 rotating brushes with a diameter of 170 mm. The brush rotates passively through the drive force from the wheel, and the mopping mechanism is pressed against the floor by a spring, to ensure that there are no gaps or gaps between the floor and the brush surface. The brush can be simply removed for cleaning or replacement.

SpO2 sensor and AMG8833 sensor

First of all, blood oxygen saturation (SpO2) was calculated using a formula supplied in the Maxim Integrated™ sample code [24]. The AC and DC

Component of the pulsative waveform was calculated for both the red and infrared channels and stored in integer variables (AC_Red, AC_IR, DC_Red, DC_IR) as a mean of 5 consecutive peaks/valleys. A ratio (R) of the AC and DC was then calculated from the mean AC and DC values using Eq. (46), and the SpO2 value was calculated by the Eq. (47):

$$R={\displaystyle \frac{A{C}_{\text{Red}}÷D{C}_{\text{Red}}}{A{C}_{\text{IR}}÷D{C}_{\text{IR}}}}$$ (46)

$$SpO2=-45,060R^2+30,345R+94,845$$ (47)

Next, the AMG8833 sensor measures the temperature of an object based on the thermal radiation emitted by that object. The Stefan-Boltzmann law describes the power emitted by an object using the formula (48):

$$P=\epsilon A\sigma T^4$$ (48)

where P is the power radiated from an object; T is the temperature of the object; A is surface area; σ is Stefan–Boltzmann constant, ε is the emissivity of the object (it gets values from 0 to <1 for objects which do not absorb all incident radiation - grey object; If it is an absolute black object then equal 1).

Applying Moghaddam's formula [31], we calculate the temperature of a grey body as follows:

$$V_{out}=k\left(T_o^4-T_s^4\right)$$ (49)

where V_out is the output voltage of the sensor, T₀ is the surface temperature of the object being measured, T_s is the temperature of the sensor's thermistor, k is the empirical constant representing the parameters A, ε, σ as well as the electronic noise that may exist during the measurement (calibrating the sensors).

Based on (49), the surface temperature of the object will be calculated by the formula (50):

$$T_o={(V_{out}/k+T_s^4)}^{1/4}$$ (50)

Experimental results

Experimenting with a robot that follows the trajectory and avoids obstacles

With the shape and size of the hospital room as shown in Fig. 18(a), we can draw a map and set the trajectory for the robot as shown in Fig. 18(b), the arrow direction is the direction of movement. The robot will both follow the trajectory and disinfect (the disinfecting road surface is 0,65 m wide), when approaching the hospital beds, the robot will collect data without contacting the patient. The robot is tested with sensor fusion cases without lidar and when lidar is present. In order to avoid obstacles appearing unexpectedly on the robot's trajectory, the Vector Field Histogram+ (VFH+) method [[21], [22], [16]] is applied. The real path of the robot is actually measured.

Fig. 18.

(a) Medical room plan; (b) The path of the robot in different configurations when following the trajectory.

In Fig. 19, the result showing the robot's path in 3 cases: the green line is the real trajectory, the red line is obtained when using only Encoder and the blue line is when adding the Lidar sensor. When using only Encoder, the robot moves away from the real trajectory when the trajectory is longer due to the accumulation of accumulated error noise, leading to the robot colliding with equipment in the room before going all the way and having to end the journey. When fusing both Encoder and Lidar sensors, the result for the robot's path is closest to the true trajectory by the positioning method based on the Kalman filter. This method is based on the data obtained from the Lidar which is matched to the global map.

Fig. 19.

The data is obtained from the Lidar and the line segments are matched to the map.

Fig. 19 shows the data obtained from the lidar sensor along the trajectory, the red lines are the lines extracted from the points and matched to the map. The Kalman filter will use the matched segments to compute the values of the matrices for the filter to locate.

Experimental results show that the effectiveness of the Kalman filter when adding more sensors, the position estimation results are closer to the real line values.

Fig. 20 provides the result of moving when the robot encounters an obstacle blocking the trajectory. The robot uses the VFH+ method to avoid obstacles based on data from ultrasonic sensors mounted on the robot. The results show that the robot avoids obstacles effectively and keeps track of its trajectory after avoiding obstacles.

Fig. 20.

Robot avoids obstacles by the VFH+ method.

Collect patient temperature and SpO2 data

Based on the robot's movement map, stop points are established along the way for the robot to stop and collect sensor data as shown in Fig. 21. We arranged the tester to lie on the bed corresponding to the stop points that will perform sensor data collection.

Fig. 21.

Map of breakpoints for robot data collection; The yellow point marks the stop position; Directional arrow for AMG8833 sensor node that collects body temperature data.

The data collection process follows the following steps: Every time the robot moves to a stop, it will perform 2 to 3 rotations at the center.

• •First, at stop position 1, the robot will rotate to the right at an angle of 45° relative to the direction of the path following the red arrow. This is the direction that is determined from the center of the image sensor to the face of the patient lying on the bed (the patient's lying position are supine, upright, and facing the ceiling). At each rotation, the robot stops for 3 s (determined through actual testing) so that the AMG8833 sensor reads the temperature data of the right patient. The next turn to the left 90° to collect the temperature of the patient on the left. In the third rotation, the robot continues to rotate left by an angle of 135° to come out of the stop and continue the route.
• •At stop position 2, the robot performs the same action as stop position 1 to collect data of 2 patients on both sides.
• •At stop position 3, the robot only performed 2 left rotations at an angle of 45° for the first time and an angle of 135° for the second time to collect data of the left patient.
• •After each rotation, the sensor data will be sent to the computer. As shown in Fig. 22, Fig. 23, we can see that every time the robot stops at a stop, the sensor data will be sent to the computer every 0,5 s. In there:
• •For the temperature graph, the red pixels represent temperatures above 35 °C, which is also the location of the tester. The group of red pixels is skewed to the right for the test participant in bed 2 and to the left for the test participant in bed 3.
• •For the SpO2 chart, every 3 s, there will be 06 data samples sent to the computer. The space between the 2 groups of data on the SpO2 and Heart Rate charts shows the time it took the robot to rotate 45° from bed 2 direction to bed 3 direction in 06 s.

Fig. 22.

Final data sample of test participants in bed 2.

Fig. 23.

Final data sample of test participants in bed 3.

Conclusions and future developments

This article presents a novel design of a mobile robot, called CEE-IMR. The research provides a smart disinfection system and indirect measurement system for mobile robots supporting Covid-19 or contagious disease treatments in hospitals. Smart disinfection robots can improve the shortcomings of existing sterilization methods, improve sterilization quality, and reduce the probability of infection. The robot model uses intelligent photo recognition, independent sensors to monitor the sterilization process in real-time, and intelligently plan moves independently. The evaluation of the results proves the effectiveness of the proposed method. Automated air and surface disinfection and non-human patient monitoring have reduced unnecessary direct contact between infected people and healthcare workers. The results can be developed for real applications in hospitals and show promise.

In the future developments, we will deploy more AI techniques and security methods into the systems. The system can operate more independently and adapt with sudden changes in flexible working environments. More facilities could be embedded in the system to provide necessary services for either patients and doctors in hospitals.

Method validation

All Results and Discussion are provided in the above Section, Method details. The results are from real robots running in hospitals that can validate the methods. The corresponding discussions are provided at every steps to validate the methods.

Limitations

The current limitation of this work is that the robot has not been trained with AI techniques to be able to adapt with either the environments or the people around. This can lead to more functions for the robot and be more effective while working in contagious disease environments.

Ethics statements

None

CRediT author statement

Hoang T. Tran, Dong TL. Tran: Conceptualization, Methodology, Software. Minh T. Nguyen, Thang C. Vu: Validity tests, Data curation, Writing- Original draft preparation. Hoang T. Tran: Visualization, Investigation. Hoang T. Tran, Thang C. Vu: Supervision. Dong TL. Tran: Software, Validation. Minh T. Nguyen, Hoang T. Tran: Writing- Reviewing and Editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

• Data will be made available on request.