Opto-ultrasound biosensor for wearable and mobile devices: realization with a transparent ultrasound transducer

Abstract

Mobile and wearable healthcare electronics are widely used for measuring bio-signals using various fusion sensors that employ photoplethysmograms, cameras, microphones, ultrasound (US) sensors, and accelerometers. However, the consumer demand for small form factors has significantly increased as the integration of multiple sensors is difficult in small mobile or wearable devices. This study proposes two novel opto-US sensors, namely (1) a wearable photoplethysmography (PPG)-US device and (2) a PPG sensor built-in mobile smartphone with a US sensor, seamlessly integrated using a transparent ultrasound transducer (TUT). The TUT exhibits a center frequency of 6 MHz with a 50% bandwidth and 82% optical transparency in visible and near-infrared regions. We developed an integrated wearable PPG-US device to demonstrate its feasibility and coupled the TUT sensor with a smartphone. We measured the heart rates optically and acoustically in human subjects and quantified the oxygen saturation optically by passing light through the TUT. The proposed proof-of-concept is a novel sensor fusion for mobile and wearable devices that require a small form factor and aim to improve digital healthcare. The results of this study can form the basis for innovative developments in sensor-based high-tech industrial applications, such as automobiles, robots, and drones, in addition to healthcare applications.

1. Introduction

Combining dissimilar sensors in a single module is increasingly researched as it improves reliability, accuracy, and various parameters in sensors by supplementing their individual limitations [1]. Particularly, mobile and wearable healthcare devices use multiple sensors to improve the quality of life of users by monitoring vital signs non-invasively [2]. Smart devices, such as smartphones and smartwatches with miniaturized built-in sensors that use cameras, photoplethysmography (PPG), and electrocardiography have emerged as portable, inexpensive, and convenient platforms for point-of-care testing.

A PPG sensor composed of a light-emitting diode (LED) and photodiode (PD) can measure the volumetric changes in blood during peripheral circulation using light attenuation properties. The PPG measurement provides beneficial biomarkers, such as heart rate, oxygen saturation, sleep trackers, and mental stress assessment [3–6]. Several studies showed that the average heart rate measured using smartphone PPG was comparable with those measured using ECGs [7–9]. Although the PPG sensor is highly advanced among mobile and wearable healthcare electronics, certain limitations are observed in terms of tissue penetration and motion artifacts [4].

Ultrasound (US) sensors are observed to compensate for these PPG sensor drawbacks [10]. US sensors are useful for non-invasively visualizing the structure of internal organs and analyzing hemodynamic functioning [11]. The transmitted US wave is partially reflected owing to the difference in mechanical properties between the tissues, while a portion of the reflected wave penetrates the tissues deeply [12]. Therefore, US sensors can measure pulsations in both superficial blood vessels in fingers and relatively deep blood vessels, including radial, brachial, and carotid arteries [10]. Additionally, US sensors exhibit superior beam directivity in contrast to LED-based PPG sensors, in addition to acquiring signals at a required point.

Recently, several groups have investigated the simultaneous use of US and PPG sensors to measure blood pulsation, flow, pressure, heart rate, and vessel diameter [13–15]. Nabeel et al. developed a bimodal US/PPG system to measure the diameter and pulse wave velocity of the carotid artery to estimate the cuffless blood pressure. Furthermore, García-López and Rodriguez-Villegas used simultaneous US M-mode imaging and PPG detection to measure the jugular vein diameter and PPG pulsation waves. These studies demonstrated that the combination of US and PPG sensors can provide significant biometric information. However, previously reported systems are complicated because the two fundamentally opaque sensors cannot be arranged coaxially. This limitation leads to significant challenges in configuring both systems with a physically small form factor. However, the complexity of the system can be reduced by overlapping the US and PPG sensors in a single housing. Additionally, more reliable and accurate multiparametric information can be obtained in the same region of interest because of coaxial alignment. This can reduce manufacturing costs and be suitable for the compact form factor of mobile medical applications.

In this study, we propose a proof-of-concept for opto-US sensors that are seamlessly integrated using a transparent ultrasound transducer (TUT): 1) a wearable PPG-US device and 2) PPG sensor built-in mobile smartphone with a US sensor. Previously, we reported the TUTs that can be seamlessly integrated with optical devices to configure a compact single unit [16–18]. In this study, we optically and acoustically measured the heart rates and oxygen saturation in human subjects to demonstrate the feasibility of TUT-based opto-US sensor devices.

2. Results

2.1. Characteristics of the TUT

Figure 1(a) depicts a photograph of the TUT. The background text written in red on the paper is clearly visible through the transparent central part of the TUT. The whole size of the TUT was 22 mm in diameter and 1 mm high. The ultrasonic active aperture, which is the central part of the TUT, was 9 mm in diameter. The fabrication process of the TUT is detailed in the Materials and Methods section. Figure 1(b) illustrates a cross-sectional diagram of the TUT. To evaluate the performance of TUT, its pulse-echo response was measured using a quartz sample in a water tank (Fig. 1(c)). The black line in Fig. 1(c) represents the pulse-echo US amplitude in the time domain, whereas the red dashed line indicates the corresponding frequency spectrum. The peak-to-peak US amplitude of the echo signal was 350 mV at 0 dB gain of the pulser-receiver (Olympus 5072PR, Olympus NDT Solutions, USA). The TUT exhibits a center frequency of 6 MHz and a –6 dB fractional frequency bandwidth of 50%. To demonstrate this, we mimicked a blood vessel using a tube with a diameter of 1 mm and measured its thickness (Fig. 1(d)). A strong US signal is observed at the tube boundary, and the diameter of the tube was measured to be about 1 mm. Figure 1(e) shows an illustrative diagram of the opto-US sensor integrating the PPG sensor and the TUT. The PPG sensor is located seamlessly on the back of the TUT. The optical beam path and US beam path are coaxially arranged through the transparent window of the TUT. For the PPG sensor, the light emitted from the LED is reflected by changes in blood volume and detected by the PD. In the case of TUT, the morphological movement of blood vessels according to the pulse is detected by transmitting and receiving US waves. Figure 1(f) depicts the experimental optical transmittance of the TUT in the wavelength range of 300–900 nm, obtained using a spectrophotometer (Cary 60 UV–Vis, Agilent, USA). The transmittance of wavelengths was more than 80% in the near-infrared region. These results indicate that PPG sensors based on visible and near-infrared wavelengths can sufficiently transmit and receive light when combined with TUT.

Fig. 1.

(a) Photograph and (b) a cross-sectional structure of a TUT. (c) Two-way pulse-echo response and its frequency spectrum of the TUT. (d) A US echo signal of the tube boundary. (e) Illustrative diagram of the opto-US sensor integrating a PPG sensor and the TUT. (f) Light transmittance of the TUT. TUT, transparent ultrasound transducer; ITO, indium-tin-oxide; LNO, lithium niobite; UV, ultra-violet; US, ultrasound; LED, light-emitting diode; PD, photodiode; PPG, photoplethysmography.

2.2. In vivo PPG/US sensing in a wearable device

To investigate the feasibility of combining PPG and TUT, we developed a wearable PPG-US device and demonstrated simultaneous sensing of wrist arterial pulses using US and PPG signals in humans in vivo. The acquisition of US signals for pulse measurement was performed in the wrist artery using the general pulse-echo method. To confirm the anatomical location of the radial artery, a US imaging system (EC-12R, Alpinion Medical Systems, Republic of Korea) with a US probe (L3-12, Alpinion Medical Systems, Republic of Korea) was used and a US B-mode image was acquired from the wrist (Fig. S1 in Supplement 1 ^{(1.9MB, pdf)} ). The wearable PPG-US device was placed on the skin above the radial artery (Fig. S2(a) in Supplement 1 ^{(1.9MB, pdf)} ) and simultaneous sensing was performed. The movement of blood vessels owing to their dilatation and constriction was detected using the TUT. The location of the anterior and posterior arterial walls could be estimated by referring to the US B-mode image. The pulsation measurement by the TUT was conducted as the movement of the anterior arterial wall. Figure 2(a) depicts a photograph of the wearable PPG-US device. The TUT is seamlessly combined with a green LED and PD in a conventional PPG sensor board. PPG and TUT were assembled in a three-dimensional printed housing and attached to the wrist via a watch strap. Figure 2(b) illustrates a schematic of the operating system of the wearable PPG-US device. The TUT was connected to the pulser-receiver, and US data were saved on a computer. We acquired both US and PPG signals simultaneously for 10 seconds from the wrists of healthy volunteers and successfully detected arterial pulsations. Owing to the vasodilation caused by the arterial pulsation, the peak of US A-line signal at the arterial boundary was periodically shifted in the time domain. The peak movement of the US A-line signal according to vasodilation in the time domain was plotted (Fig. 2(c)). Furthermore, PPG signals were continuously acquired through the PD in synchronization with the US pulse repetition interval (Fig. 2(d)). The peaks of the US and PPG signals were displayed simultaneously and the interval time of the peak of the US and PPG signals were identical, respectively. In addition, frequency components of US and PPG signals were analyzed using a fast Fourier transform to calculate the heart rate. Both US and PPG signals exhibited a fundamental frequency of 1 Hz, which corresponds to 60 bpm.

Fig. 2.

(a) Photographs of a wearable PPG-US device and close-up of the back-side of the device. (b) Schematic of the device. Simultaneous sensing of (c) US and (d) PPG signals in humans in vivo and their frequency spectra. PPG, photoplethysmography; TUT, transparent ultrasound transducer; LED, light-emitting diode; PD, photodiode.

2.3. In vivo PPG/US sensing in a mobile device

To further investigate the capability of opto-US sensor devices, we combined TUT with a PPG sensor embedded in a smartphone. Typically, a PPG sensor with two LEDs inside a smartphone is used for routine measurement of individual bio-signals, such as the heart rate and oxygen saturation. The TUT was manually coupled over a window where the PD and LEDs were visible on the back of the smartphone (Fig. 3(a)). Similar to the wearable PPG-US device, TUT collected US echo signals from the artery on the wrist (Fig. S2(b) in Supplement 1 ^{(1.9MB, pdf)} ) and saved on the computer. PPG signals were transferred from the smartphone to the computer immediately after data acquisition was completed (Fig. 3(b)). Supplementary material, Visualization 1 ^{(14.5MB, mp4)} is a conceptual demonstration video of the simultaneous measurement of US and PPG signals on a TUT-mounted smartphone. The raw US A-line signal acquired in real-time was displayed in the digital oscilloscope (DPO 3032, Tektronix, USA) and the PPG signal was displayed on the smartphone. The US A-line signal of artery wall shifted momentarily when the blood vessels vasodilated. Additionally, the average oxygen saturation was measured to be 97% owing to the presence of two built-in LEDs in the PPG sensor of the smartphone shown in Visualization 1 ^{(14.5MB, mp4)} . Figures 3(c) and (d) depict the results measured for 10 seconds from the US and PPG signals, respectively. The PPG signal was acquired using two LEDs simultaneously to measure not only heart rate but also oxygen saturation. As indicated in the figures, the peak interval times of both US and PPG signals are identical. Each signal was analyzed using fast Fourier transformation to calculate the heart rate based on the US and PPG data. The fundamental frequency of the acquired US and PPG signals was determined to be 0.95 Hz, which corresponds to 57 bpm. The calculated bpm values from the US and PPG signals concurred with the values displayed in the smartphone application.

Fig. 3.

(a) Photograph of a PPG sensor and pulse oximeter embedded in a mobile smartphone with TUT and a close-up of the back of the smartphone; (b) the corresponding schematic. Simultaneous sensing of (c) US and (d) PPG signals in humans in vivo and their frequency spectra. PPG, photoplethysmography; TUT, transparent ultrasound transducer; LED, light-emitting diode; PD, photodiode

3. Materials and methods

3.1. TUT fabrication process

The TUT piezoelectric layer is composed of a single-crystal lithium niobate disk (LNO, Boston Piezo-Optics Inc., USA) with a thickness of 250 µm. Additionally, 200-nm-thick indium tin oxide (ITO) thin films were used as transparent electrodes for executing the piezoelectric effect of the LNO layer. A 150-nm-thick donut-shaped gold film was deposited on the outer aluminum tube and the lower surface of the LNO served as an electrical bridge. Therefore, the inner and outer aluminum tubes serve as conductive fixtures for actuating the TUT by applying a voltage to the upper and lower surfaces of the LNO, respectively. We used Epotek301, a non-conductive epoxy, to dampen vibrations and insulate the inner and outer tubes. Subsequently, two matching layers were configured to improve the ultrasonic transmission and reception efficiency of the TUT. A 100-µm-thick cover-glass was carefully cured under the ITO electrode by applying a colorless ultra-violet epoxy resin. Additionally, parylene C was vapor-coated on the cover-glass for electrical insulation, protection, and impedance matching using soft tissues.

3.2. Wearable PPG/US sensor device

For PPG sensing, the printed circuit board (PCB) consisting of LED (VLMTG1400, Vishay Semiconductors, PA, USA) and PD (SFH2716, Osram Opto Semiconductors, Regensburg, Germany) was prepared. To acquire PPG and US signals at the same position, the center of the TUT was aligned to be located at the midpoint of the LED and PD. Then, the PCB with the TUT was embedded in a custom-made watch-type structure with a commercial watchstrap to minimize the motion of subjects during experiments. Through the TUT, the light from the LED was transmitted to the body, and the returned light was measured by the PD. The small current converted from the PD was amplified by a transimpedance amplifier (OPA2380, Texas Instruments, TX, USA) and a voltage amplifier (ADA4522, Analog devices, MA, USA), and acquired on the analog ports in a multifunctional I/O device (PCIe-6321, National Instruments, TX, USA), which were built in the PCB. PPG signals were acquired at a sampling rate of 62.5 Hz. The electronics in the PCB were activated by a voltage source supplied by power equipment (DP30-05TP, TOYOTECH, Republic of Korea). For US sensing, the TUT was connected to a pulser-receiver (5073PR, Olympus NDT, MA, USA) to generate short US pulses with 10 kHz repetition rate and amplify returned echoes with 20 dB gain. The received US A-line signals from the TUT were digitalized via a waveform digitizer (ATS9350, Alazar Technologies, QC, Canada) with a sampling rate of 250 MS/s. To synchronize PPG and US signals, the sequence for triggering was customized using the digital ports in the I/O device and the program coded on LabVIEW (National Instruments, TX, USA). The PPG board consumes 0.15 W and the pulser-receiver consumes 10 W.

3.3. Mobile PPG/US sensor device

A Galaxy S8 smartphone (Samsung Electronics, Republic of Korea) was utilized for demonstrating the mobile PPG/US sensor device. Galaxy S8 contains a PPG sensor called a heart rate sensor. The PPG sensor with a PD and two LEDs is located on the back of the Galaxy S8. The two LEDs have peak intensities at wavelengths 660 nm and 885 nm, respectively (Fig. S3 in Supplement 1 ^{(1.9MB, pdf)} ). The TUT was placed carefully and temporarily with the black insulation tape on the back of the Galaxy S8 with the PPG sensor exposed. The two LEDs and the PD of the PPG sensor were optically exposed to the transparent window, which is the active aperture of the TUT. As similar as the wearable PPG/US sensor device, the lights from the LEDs were transmitted through the TUT, and received by the PD at a sampling rate of 100 Hz. In addition, US signal is detected with the same principle of operation using the pulser-receiver. To extract the raw PPG signal from the Galaxy S8, a smartphone app from Google Play called Heart rate analyzer was used.

3.4. Experimental protocol

All procedures involving human subjects were performed following the regulations and guidelines approved by the Institutional Review Board of Pohang University of Science and Technology. We recruited healthy volunteers to acquire US and PPG signals from the wrist. The participants were provided sufficient information regarding the experiment before its execution. For both wearable and mobile PPG/US sensor devices, a small amount of US gel was applied under the TUT for effective US data acquisition. PPG and US data were collected for 10 seconds under normal conditions after participants had adequate rest. To reduce unnecessary noise due to motion, the participants remained stationary during the experiment.

4. Discussion

This study proposes a novel opto-US sensor fusion of PPG and TUT for analyzing human bio-signals using wearable and mobile devices. Owing to the optical transparency of TUT, the PPG sensor can be coaxially combined with the TUT seamlessly. To the best of our knowledge, this is the first study to combine PPG sensors with TUT. The specifications of TUT can be briefly summarized as follows: 6 MHz center frequency, –6 dB fractional bandwidth of 50%, and optical transmittance of more than 80% in the visible and near-infrared regions. We simultaneously acquired PPG and US signals in real-time from the blood vessels of the human wrist using opto-US sensors, namely the wearable PPG-US device and PPG sensor-US device in the mobile smartphone. The frequency analyses of the acquired PPG and US signals verify that the calculated heart rates concur with each other. In particular, the opto-US sensor module combined with TUT performs well in all green (532 nm), red (660 nm), and near-infrared (885 nm) wavelengths, demonstrating the robustness of the results. These results validate that the opto-US sensor fusion device can be effectively used by delivering multi-parametric information via wearable and mobile devices with a small form factor. However, the proposed opto-US sensor device for TUT requires further investigations. The accurate measurement of the arterial thickness is challenging with our current TUT due to the weak signal-to-noise ratio and ring-down artifacts. We agree that accurate measurement of arterial thickness is difficult with our current TUT. The main ways to improve the performance of the current TUT are as follows: 1) It is necessary to develop a multi-element TUT. Multi-element TUTs can transmit and receive multiple US signals simultaneously and focus through beamforming. 2) The backing layer of the TUT can be optimized to reduce ring-down artifacts. By reducing the ringdown, the posterior artery wall can be clearly distinguished. Based on these discussions, future research will be able to provide various information such as thickness, area, and blood flow measurement of arteries using US signals. Another issue is that the housing of the current TUT is relatively large for applications in mobile healthcare devices. To address this problem, the TUT housing should be optimized with minimal size, considering the size of the PPG sensor module. Additionally, PPG and TUT should be combined in a single housing to form a miniaturized opto-US sensor device. The electrical cables of the TUT being independently exposed can be addressed using a customized printed circuit board with wireless power and data transceiver modules rather than electrical cables. The PCB-based opto-US sensors can be configured for easy application in various types of mobile devices. Moreover, the pulser-receiver is essential equipment to operate the US transmission/reception, however, the pulser-receiver used in this study is relatively large equipment made for research purposes. Recently, with the rapid development of the semiconductor field, the pulser-receiver is being commercialized as hand-sized miniaturized equipment as a hand-sized small device. These technological advances are expected to enable miniaturization for use in wearable and mobile devices in the near future. Accordingly, TUT can be applied in both PPG-US sensing and optical US fusion-based technologies in different fields, such as medical imaging [19–23], disease treatment [24,25], non-destructive testing [26,27], autonomous driving [28], and fingerprint recognition [29]. We expect that TUT-based opto-US devices can be game-changers for innovations in sensor-based high-tech industries in the future.

Funding

Ministry of Education10.13039/501100002701 (2020R1A6A1A03047902, 2021R1A6A3A13044749); Ministry of Science and ICT, South Korea10.13039/501100014188 (2019R1A2C2006269, 2020R1C1C1013549, 2021M3C1C3097619); Korea Medical Device Development Fund10.13039/100019266 (9991007019, KMDF_PR_20200901_0008); BK21 FOUR project.

Disclosures

C. Kim and J. Y. Kim have financial interests in OPTICHO, which, however, did not support this work.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 ^{(1.9MB, pdf)} for supporting content.