KUBAI: Sensor Fusion for Non-Invasive Fetal Heart Rate Tracking

Abstract

Objective:

Fetal heart rate (FHR) is critical for perinatal fetal monitoring. However, motions, contractions and other dynamics may substantially degrade the quality of acquired signals, hindering robust tracking of FHR. We aim to demonstrate how use of multiple sensors can help overcome these challenges.

Methods:

We develop KUBAI¹, a novel stochastic sensor fusion algorithm, to improve FHR monitoring accuracy. To demonstrate the efficacy of our approach, we evaluate it on data collected from gold standard large pregnant animal models, using a novel non-invasive fetal pulse oximeter.

Results:

The accuracy of the proposed method is evaluated against invasive ground-truth measurements. We obtained below 6 beats-per-minute (BPM) root-mean-square error (RMSE) with KUBAI, on five different datasets. KUBAI’s performance is also compared against a single-sensor version of the algorithm to demonstrate the robustness due to sensor fusion. KUBAI’s multi-sensor estimates are found to give overall 23.5% to 84% lower RMSE than single-sensor FHR estimates. The mean±SD of improvement in RMSE is 11.95±9.62BPM across five experiments. Furthermore, KUBAI is shown to have 84% lower RMSE and ~3 times higher ${\mathrm{R}}^2$ correlation with reference compared to another multi-sensor FHR tracking method found in literature.

Conclusion:

The results support the effectiveness of KUBAI, the proposed sensor fusion algorithm, to non-invasively and accurately estimate fetal heart rate with varying levels of noise in the measurements. Significance: The presented method can benefit other multi-sensor measurement setups, which may be challenged by low measurement frequency, low signal-to-noise ratio, or intermittent loss of signal in measurements.

I. Introduction

Fetal heart rate (FHR) is the fundamental to assessing the well-being of the fetus throughout pregnancy and labor. By analyzing the temporal relationship between FHR and mother’s uterine contractions, obstetricians classify FHR patterns as ”reassuring” or ”nonreassuring” [2]. Cardiotocography (CTG) is the current technology used to monitor FHR with a Doppler ultrasound transducer placed on the maternal abdomen [3]. Unfortunately, CTG is very prone to noise sources such as fetal motion, maternal and fetal breathing, and the placement of the ultrasound transducer requires frequent adjustment [4], [5]. In addition to CTG, researchers have been working on developing a Transabdominal Fetal Pulse Oximetry (TFO) device to provide non-invasive and transcutaneous measurement of fetal oxygen saturation [6]–[10]. TFO aims to support obstetricians in objectively assessing the fetal well-being. In order to compute the oxygen saturation, TFO relies on extraction of fetal heart rate from the acquired photoplethysmogram (PPG) signals [6], [7]. Thus, FHR is a precursor to TFO’s fetal oxygen saturation computation. Alas, fetal PPG signals captured using TFO have a very low signal-to-noise ratio (SNR). Fetal signal losses can occur due to motion or TFO’s placement, and FHR can be masked by maternal heart rate or maternal respiration rate [11], [12]. It is crucial to overcome the challenges of reliable data collection that are due to unforeseeable dynamics of the physical world in order to improve the accuracy of fetal monitoring technology. Non-deterministic and probabilistic methods are necessary.

The use of multiple sensors to prevent data losses or reduce noise contribution has been largely employed in many different application areas [13]–[16]. By combining data from multiple sensors, one can achieve higher accuracy and can make more specific deductions compared to single source data [17].

We propose KUBAI¹, a situation-aware algorithm that makes use of stochastic multi-sensor fusion algorithm to improve tracking a very noisy physiological signal. We use data collected from gold standard animal models using Transabdominal Fetal Pulse Oximeter to demonstrate the improvement in FHR extraction accuracy with the integration of sensor fusion. This paper accomplishes accurate FHR monitoring by performing the following:

• First, we review how fetal signal can be best filtered from other physiological noise sources to improve the analysis of sensory measurements.

• Next, we introduce the particle filtering algorithm that increases the robustness against unforeseeable dynamics of the physical world thanks to its stochastic nature. We further improve the particle filtering algorithm’s performance by incorporating sensor fusion. The resulting algorithm is named KUBAI.

• Finally, the proposed approach is evaluated on data captured from gold standard animal models using TFO. The performance of KUBAI is compared against the single-sensor version of the algorithm, and also bench-marked against a previously published method.

II. Background

A. Transabdominal Fetal Pulse Oximeter

Obstetricians currently rely on monitoring temporal relationship of fetal heart rate with uterine contractions to assess correct oxygen supply to the fetus. This method has proven to have a high false positive rate, causing an increase in unnecessary C-Sections performed in the US [19], [20]. The non-invasive Transabdominal Fetal Pulse Oximeter (TFO) aims to lower false fetal hypoxia diagnosis [8], [21].

TFO is comprised of an optical probe (optode), shown in Fig. 1a, designed to house two high-power Near-Infrared (NIR) LEDs with wavelengths at 740nm and 850nm. In our application, the radiated power is limited to <500mW for 850nm, and to <350mW for 740nm, to prevent thermal hazard to the skin. Moreover, the LEDs are pulsated at frequencies beyond flicker noise region with 50% duty-cycle. This further reduces heat dissipation. A temperature sensor placed near the LEDs continuously monitors the skin temperature, to ensure it does not increase beyond 3°C above normal body temperature.

Fig. 1:

(a) Picture of Optical Probe (Optode) [11]. (b) Illustration of Transabdominal Fetal Pulse Oximetry Principle [18].

As shown in Fig. 1b, TFO is a reflectance pulse oximeter with the optode placed on the maternal abdomen. The detector placement on the optode is based on previous publications simulating the physics of light-tissue interaction to optimize the reception of fetal signal up to 5cm depth [12], [18], [21].

Photocurrent produced by photodetectors on the optode is converted to voltage using a transimpedance amplifier. The electrical voltages are further amplified and sampled using a 24-bit analog-to-digital converter with built-in programmable-gain amplifier (PGA). A custom embedded optode control system adjusts the LED drive current and modulation frequency, as well as the PGA gain. A custom real-time software eases communication with the embedded control system, plots data real-time, and saves it for post-processing [7].

The closest photodetector to the light source with 1.5cm distance (D1 in Fig. 1a) receives photons that only scatter through maternal tissues, and thus can be used to find maternal heart rate (MHR) and maternal oxygen saturation (MSpO₂).

As the LED-photodetector distance increases, the reflected light travels deeper into the abdomen. Therefore, the further photodetectors with 3, 4.5, 7, and 10 cm distance from the LEDs (D2 to D5 in Fig. 1a), can collect a mixed information from both maternal and fetal layers [18], [22]. The fetal signal needs to be separated from the mixed signal, and further processed using conventional pulse oximetry algorithms to compute fetal oxygen saturation (FSpO₂) [22], [23].

B. Fetal Signal Separation and Analysis

1). Mixed Signal Problem and Noise Sources:

The light detected at the far detectors (D2 to D5) is a mixed signal containing photons that travel through both maternal and fetal tissues. The maternal portion of the mixed signal is considered noise and needs to be filtered to expose the fetal PPG. The maternal noise includes periodic signals such as maternal heart rate (MHR) (60–120 beats-per-minute (BPM)), maternal respiration rate (MRR) (12–20 breaths per minute) and Mayer Waves (~0.1Hz). Other sources of noise are electronic noise of the system and motion [7].

The fetal signal’s desired component is the fetal heart rate (FHR) (110–270 BPM) which is used in FSpO₂ computation [7], [23]. The fetal heart rate’s upper limit is defined as 270BPM to account for heart rate increase due to hypoxia.

All these noise sources can cause losing fetal signal in the PPG. Fig. 2 shows an example spectrogram of a mixed-signal PPG where motion caused increase in noise-floor, masking the harmonic signals. The maternal noise contribution is a large contributor to noise since it is consistently present in the measurements, unlike randomly occurring motion noise. Maternal noise can mask the FHR if the MHR or MRR harmonics lie close to FHR. Fig. 2 shows a situation where the maternal heart rate’s second harmonic overlaps with FHR, completely masking it. Therefore, it is critical to filter the maternal noise contribution.

Fig. 2:

Example Spectrogram Showing Loss of Fetal Heart Rate due to Motion and Maternal Heart Rate.

2). Signal Separation:

Adaptive noise cancellation (ANC) is a well performing algorithm when it comes to filtering the maternal noise and revealing the fetal signal from the mixed signal [11], [18], [22]. The reference maternal noise required for ANC algorithm is captured by the nearest detector D1 to the LED emitters. ANC is applied to all four further detectors (D2 to D5), which can capture a mixed signal depending on the depth of the fetus. As a result, we obtain four separate ANC filtered fetal signal measurements thanks to the multi-sensor design of the system.

3). Signal Analysis:

It is best to analyze TFO measurements in frequency domain due to the periodic nature of physiological signals. Additionally, frequency domain analysis has the advantage of improving signal-to-noise ratio (SNR) thanks to windowing and time averaging.

III. Problem statement

Fetal hart rate tracking from transabdominally acquired PPG data faces multiple challenges. First, the sensed PPG data has weak fetal contribution due to the long path the light has to travel before it can reach the fetus. This causes higher light absorption and lower signal-to-noise ratio (SNR) of the desired signal. Second, human physiology varies greatly among people and over time creating inter- and intra-experiment variability. Third, the TFO measurements suffer from frequent signal loss due to unforeseeable dynamics of the physical world and noise sources presented in Section II-B. Lastly, both the fetal signal and the noise signals have a non-stationary nature over time. All these challenges present a threat to reliable fetal heart rate estimation for TFO. Thus, this calls for stochastic methods to more accurately track fetal heart rate using sensory data collected from a very dynamic physical medium.

The main problem this paper addresses is, how can we accurately estimate fetal heart rate given multiple noisy PPG datasets captured simultaneously using five photodetectors present on TFO.

IV. Multi-sensor fetal heart rate tracking

A. Particle Filtering for State Estimation

We want to recursively estimate the FHR state $X_{1:t}$ over time, using the PPG measurements $Y_{1:t}$ (a.k.a observations). We start by making a large number of random hypotheses on what the FHR can be. These hypotheses are called particles. The particle filter evaluates how likely each particle is to be equal to FHR, based on the measurement we have in hand. If a particle is very likely to represent the true FHR, then we assign a high weight to it. Particles with larger weights contribute more to the output of the particle filter. Eventually, the output of the particle filter is expected to converge, from the initial random hypotheses, towards FHR thanks to the measurements, and particle weight assignment based on likelihood [24], [25].

Particle Filtering views the state space as a Hidden Markov Model described by two distributions: (1) Prior distribution $p\left(x_t|x_{t-1}\right)$, (2) Measurement likelihood $p\left(y_t|x_t\right)$. We assume each observation $Y_t$ only depends on current state $X_t$ [25].

Weights per particle $\left(i\right)$ are computed recursively using a chosen prior distribution $p\left(x_t|x_{t-1}\right)$, measurement likelihood $p\left(y_t|x_t\right)$ and proposal distribution $q\left(x_t|x_{1:t-1},y_{1:t}\right).{\tilde{W}}_t^{\left(i\right)}$ and $W_t^{\left(i\right)}$ are the non-normalized and normalized particle weights and are defined as in (1) and (2) [24].

$${\tilde{W}}_t^{\left(i\right)}={\tilde{W}}_{t-1}^{\left(i\right)}\frac{p(y_t|X_t^{\left(i\right)})p(X_t^{\left(i\right)}|X_{t-1}^{\left(i\right)})}{q(X_t^{\left(i\right)}|X_{1:t-1}^{\left(i\right)},y_{1:t})}$$ (1)

$$W_t^{\left(i\right)}=\frac{{\tilde{W}}_t^{\left(i\right)}}{{\sum }_{i=1}^N{\tilde{W}}_t^{\left(i\right)}}$$ (2)

The proposal distribution $q\left(x_t|x_{1:t-1},y_{1:t}\right)$ is commonly chosen to be equal to prior distribution. This simplifies the particle weight computation defined in (1) to

$$\begin{gathered} {\tilde{W}}_t^{\left(i\right)}={\tilde{W}}_{t-1}^{\left(i\right)}p(y_t|X_t^{\left(i\right)}) \\ \text{if}q\left(x_t|x_{1:t-1},y_{1:t}\right)=p\left(x_t|x_{t-1}\right) \end{gathered}$$ (3)

As we recursively compute the particle weights and estimate the FHR state $x_k$ given PPG observations $y_k$, some weights become negligible over time. Particles with negligible weights need to be replaced. Systematic resampling is employed once the effective number of particles $N_{eff}$, estimated using (4), drops below a set threshold $N_{\mathit{thr}}$ [25].

$${\hat{N}}_{e}ff=\frac{1}{{\sum }_{i=1}^N{(W_t^{\left(i\right)})}^2}$$ (4)

To summarize, the parameters that define a particle filter are: (1) Number of particles $\mathrm{N}$, (2) Initial particle distribution $p\left(x_1\right)$, (3) Prior distribution $p\left(x_t|x_{t-1}\right)$, (4) Measurement likelihood $p\left(y_t|x_t\right)$, (5) Proposal distribution $q\left(x_t|x_{1:t-1},y_{1:t}\right)$, and (6) Effective particle threshold $N_{\mathit{thr}}$ to trigger resampling.

Once these parameters are defined, a particle filter with $i=1,\dots ,N$ particles iterates through the following steps [25],which are visualized in Fig. 3 in the context of FHR estimation:

Fig. 3:

Simplified Overview of Particle Filtering for Estimating Fetal Heart Rate using TFO Measurements.

1. Initially sample $X_1^{\left(i\right)}\sim p\left(x_1\right)$, otherwise sample $X_t^{\left(i\right)}\sim q(x_t|X_{1:t-1}^{\left(i\right)},y_{1:t})$.

2. Compute and normalize the weights $W_t^{\left(i\right)}$ using (1), (2).

3. Output state estimation ${\hat{x}}_t$ using $\{W_t^{\left(i\right)},X_t^{\left(i\right)}\}$. The state estimation is equal to the mean of all weighted particles.

4. If ${\hat{N}}_{e}ff<N_{thr}$ resample $\{W_{1:t}^{\left(i\right)},X_{1:t}^{\left(i\right)}\}$ to obtain $\mathrm{N}$ equally-weighted particles $\{\frac{1}{N},X_{1:t}^{\left(i\right)}\}$.

B. Spatial Sensor Fusion and KUBAI

Measurement likelihood $p\left(y_t|x_t\right)$ definition is key to good particle weight assignment $W_t^{\left(i\right)}$, and accurate state estimation.

In TFO, we have multiple photodetectors, thus a K dimensional measurement space $y_t\in R^K$. Individual photodetectors can generate unreliable measurements due to noise sources described in Section II-B. By defining a measurement likelihood $p\left(y_t|x_t\right)$ that fuses individual measurement likelihoods $p(y_t^{\left(k\right)}|x_t)$, we can improve the accuracy of state estimation by increasing robustness against noise and fetal information losses occurring in individual detectors.

We have prior knowledge on reliability ${\lambda }_k$ of TFO’s individual photodetector measurements based on light source-detector distance. Previous work have shown that photodetectors placed further away from the LED light source capture photons that penetrated deeper into the tissue [12]. Thus, photodetectors with larger source-detector distance capture more photons that have travelled through fetal layers giving higher fetal sensitivity. Unfortunately, the cost for higher fetal sensitivity is to capture an overall weaker signal because light traveling deeper will also be absorbed more and reflected less to photodetectors on the surface. Therefore, photodetectors with large source-detector distance capture both more fetal signal and less overall signal [18].

We use weighted-sums approach to define $p\left(y_t|x_t\right)$, and leverage this knowledge on photodetector measurement reliability to define the weight ${\lambda }_k$ per $p(y_t^{\left(k\right)}|x_t)$ as seen in (5). The resulting algorithm is named KUBAI. KUBAI makes use of particle filters to fuse data from multiple sensors and consequently, yields a spatially-aware estimate of FHR.

$$p\left(y_t|x_t\right)=\sum _{k=1}^K{\lambda }_k*p(y_t^{(k)}|x_t)$$ (5)

C. State Estimation with Missing Measurements

As explained in section II-B, it is best to analyze TFO measurements in frequency domain to better recover fetal heart rate information. Due to low signal-to-noise ratio (SNR) of fetal signal, one cannot use a very short time window when computing the signal’s Fast Fourier Transform (FFT). Longer FFT window length results in lower noise floor in frequency domain thanks to averaging. If our algorithm depended on the FFT measurement rate, then it would suffer from low time resolution due to long windows. Fortunately, since KUBAI makes use of particle filtering, it does not solely depend on measurement $y_t$ to estimate the state $x_t$ at any point in time.

If the proposal distribution $q\left(x_t|x_{1:t-1},y_{1:t}\right)$ is chosen to be equal to prior distribution $p\left(x_t|x_{t-1}\right)$, then the algorithm can keep estimating the next state in time and only compute the particle weights when a measurement becomes available. Once a prior distribution is defined, the state estimation can be outputted more frequently while adjusting the weights less frequently, only when a new measurement becomes available.

Note that when a measurement is made available to KUBAI at time $\mathrm{t}$, it refers to a window of measurements that are collected from $t-L/2$ to $t+L/2$, where $L$ is the FFT window length. Thus, KUBAI has a latency of $L/2$ relative to real-time.

D. KUBAI’s Particle Filter Parameters

This section details the definition of KUBAI’s parameters, used in the workflow of the algorithm shown in Fig. 4. In our application, the FHR state depends on physiological changes which are challenging to model. Prior knowledge and trade-offs are considered when defining each parameter.

Fig. 4:

Workflow of KUBAI: Particle Filter with Sensor Fusion

1). Prior Distribution $\mathbf{p}({\mathbf{x}}_{\mathbf{t}}|{\mathbf{X}}_{\mathbf{t}-\mathbf{1}}^{\left(\mathbf{i}\right)})$:

In KUBAI, the prior distribution needs to represents how FHR evolves in 1 second. We know that: (1) FHR of a healthy fetus is expected to vary by <5BPM in 1 second, (2) Hypoxia causes increase in FHR.

A Gaussian distribution is generic enough to represent the FHR motion over time, with an equal probability to both increase and decrease the FHR. A Poisson distribution can better model the increase in FHR caused by hypoxia, but would not generalize well to non-hypoxia cases. A fusion of the two distributions best suits our application.

We sample N particles from a Gaussian distribution centered at previous FHR state per particle $X_{t-1}^{\left(i\right)}$, and with a variance ${\sigma }^2=5BP{M}^2$, as defined in (6). We sample an additional $\mathrm{N}$ particles from a Poisson distribution with a rate variable $\lambda =0.1BPM$, offset by previous FHR state per particle $X_{t-1}^{\left(i\right)}$, as defined in (7). Finally, we randomly select $\mathrm{N}$ particles out of the set of $2\mathrm{N}$ particles sampled from both Gaussian and Poisson distributions. This way, we boost the probability of sampling particles with an increasing FHR, while still having some probability of sampling particles with a decreasing FHR.

$$\begin{gathered} p_1(x_t|X_{t-1}^{\left(i\right)})=𝒩(\mu =X_{t-1}^{\left(i\right)},{\sigma }^2=5) \\ =\frac{1}{\sqrt{5*2\pi }}{e}^{-\frac{1}{2}{\left(\frac{x_t-X_{t-1}^{\left(i\right)}}{\sqrt{5}}\right)}^2} \end{gathered}$$ (6)

$$p_2(x_t|X_{t-1}^{\left(i\right)})=X_{t-1}^{\left(i\right)}+\frac{{\lambda }^{x_t}{e}^{-\lambda }}{x_t!}$$ (7)

2). Measurement Likelihood $\mathbf{p}({\mathbf{y}}_{\mathbf{t}}|{\mathbf{X}}_{\mathbf{t}}^{\left(\mathbf{i}\right)})$:

We explained in Section IV-B that by employing sensor fusion in the measurement likelihood, we can increase the robustness of FHR estimation against signal losses in individual sensor measurements. The measurement likelihood needs to be defined for individual sensor measurements $p(y_t^{\left(k\right)}|X_t^{\left(i\right)})$ and then fused using the weighted-sums approach as shown in (5).

The measurements input to KUBAI are, as detailed in Section V-B, power spectrums per detector $Y_t(f{)}^{\left(k\right)}$, with peaks present at FHR, and first two MHR harmonics that fall inside the FHR range. Beside the estimated FHR state, the actual state of the system includes both MHR and FHR. Therefore, reference MHR readings $Z_t$ are input as an additional variable to the measurement likelihood $p(y_t|X_t^{\left(i\right)},Z_t)$.

The likelihood of a detector $\mathrm{k}$ having a good FHR measurement should be high when a given particle’s FHR state $X_t^{\left(i\right)}$ and MHR state $Z_t$ match with peaks in the measured power spectrum $Y_t(f{)}^{\left(k\right)}$. This can be seen as a reward function $f_{\mathit{reward}}(Y_t^{\left(k\right)}|X_t^{\left(i\right)},Z_t)$ because the more peaks in $Y_t^{\left(k\right)}$ match with $\{X_t^{\left(i\right)},Z_t\}$, the more those particles should be rewarded. We define $f_{\mathit{reward}}(Y_t^{\left(k\right)}|X_t^{\left(i\right)},Z_t)$ as the area of power spectrum multiplied by sum of Gaussians located at $X_t^{\left(t\right)}$ and $Z_t$. The Gaussian distribution centered at $X_t^{\left(i\right)}$ is set to have a variance of $1BP{M}^2$ and the Gaussian distribution centered at $Z_t$ is set to have a variance of $0.25BP{M}^2$. The Gaussian centered at $X_t^{\left(i\right)}$ has a higher variance because FHR is more variable over time than the MHR of an anesthetized ewe. The equation for $f_{\mathit{reward}}(Y_t^{\left(k\right)}|X_t^{\left(i\right)},Z_t)$ is given in (8). Fig. 5 illustrates the reward function applied to a single particle and detector measurement for ease of understanding.

Fig. 5:

Illustration of Reward Function $f_{\mathit{reward}}(Y_t^{\left(k\right)}|X_t^{\left(i\right)},Z_t)$ Computed using Single-Detector Measurement $Y_t^{\left(k\right)}$.

$$\begin{gathered} f_{\mathit{reward}}(Y_t^{(k)}|X_t^{\left(i\right)},Z_t)={\int }_0^{f_s/2}{Y}_t(f{)}^{\left(k\right)}*( \\ \left.\frac{1}{2}𝒩(X_t^{\left(i\right)},1)+\frac{1}{4}𝒩\left(Z_t,0.25\right)+\frac{1}{4}𝒩\left(2Z_t,0.25\right)\right)df \end{gathered}$$ (8)

To find the total reward function $f_{\mathit{reward}}(y_t|X_t^{\left(i\right)},Z_t)$, we fuse individual detector’s reward functions using the weighted-sums approach, as shown in (9).

$$\begin{gathered} f_{reward}(Y_t|X_t^{(i)},Z_t)=\sum _{k=1}^K{\lambda }_k*f_{reward}(Y_t^{(k)}|X_t^{(i)},Z_t) \\ \text{with}{\lambda }_k=\left\{\frac{1}{8},\frac{2}{8},\frac{3}{8},\frac{2}{8}\right\} \end{gathered}$$ (9)

The reward function $f_{\mathit{reward}}(Y_t|X_t^{\left(i\right)},Z_t)$ is then fit to a Sigmoid to define the measurement likelihood $p(Y_t|X_t^{\left(i\right)},Z_t)$, as shown in (10). The advantage of having $p(Y_t|X_t^{\left(i\right)},Z_t)$ defined as a sigmoid function is the ability to create an inflection point that can quickly kill particles with bad FHR estimates and reward particles with better FHR estimates. It also does not differentiate between similar performing particles by saturating the maximum probability after the inflection point. The parameters $\alpha$ and $\beta$ of the sigmoid function are found by performing a grid search and maximizing the accuracy of KUBAI’s FHR estimates in one experiment dataset.

$$\begin{gathered} p(Y_t|X_t^{\left(i\right)},Z_t)=\frac{1}{1+e^{\alpha (-f_{\mathit{reward}}(Y_t|X_t^{(i)},Z_t)+\beta )}} \\ \text{with}\alpha =470,\beta =4.9e-3 \end{gathered}$$ (10)

3). Proposal distribution $\mathbf{q}({\mathbf{x}}_{\mathbf{t}}|{\mathbf{X}}_{\mathbf{1}:\mathbf{t}-\mathbf{1}}^{\left(\mathit{i}\right)},{\mathbf{Y}}_{\mathbf{1}:\mathbf{t}})$:

We choose the proposal distribution $q(x_t|X_{1:t-1}^{\left(i\right)},Y_{1:t})$ to be equal to prior distribution $p(x_t|X_{t-1}^{\left(i\right)})$ which simplifies the particle weight computation to (3).

4). Threshold ${\mathbf{N}}_{\mathbf{thr}}$:

The threshold $N_{\mathit{thr}}$ of effective number of particles is set to 75%. Once less than 75% of particles are effectively contributing to the state estimation output, systematic resampling is triggered [25].

Now that all particle filter parameters of the KUBAI algorithm are defined, the properly weighted particles can be readily used to output the estimated fetal heart rate state. We output the average of weighted particles as our FHR estimate.

V. Experimental results and discussion

A. In Vivo Data Collection

The data used to evaluate the performance of the proposed approach is collected from gold standard hypoxic lamb model. All procedures followed the protocol approved by UC Davis Institutional Animal Care and Use Committee (IACUC) [9].

Three near-term pregnant ewes were put under anesthesia and an aortic occlusion balloon catheter was inserted to induce fetal hypoxia gradually. Reference FHR data was collected via hemodynamics through an arterial line inserted into the fetus’ neck. After the insertion of the arterial line, the fetus was returned to the uterus and the incision was sutured. Reference MHR values were captured using a conventional pulse oximeter on the ewe’s abdomen. The TFO optode, shown in Fig. 1a, was secured in place on the ewe’s abdomen and good skin contact was ensured. These experiments were conducted for a different study and more details about the setup can be found in [7].

B. Measurement Data preparation for KUBAI

Since FHR is easier to track in frequency domain, the measurements $Y_t$ input to KUBAI are power spectrums of time-series photodetector data. There are multiple data processing steps before measurements are made available to KUBAI. All processing steps are visualized in Fig. 8.

Fig. 8:

Block Diagram of Data Processing Steps for KUBAI.

1). Time Domain Bandpass Filtering:

We first apply a zero-phase IIR bandpass filter to the time-series PPG measurements to remove DC and high frequency noise signals. The passband is [0.2Hz, 15Hz] (or [12BPM, 900BPM]). It is important to apply zero-phase filtering to avoid phase-distortion.

2). Maternal Noise Cancellation:

As seen in section II-B, MHR (60–120BPM) and its harmonics can lie close to or even overlap with FHR (110–270BPM), therefore it is crucial to cancel its presence in the mixed signal as much as possible.

Recursive-Least-Squares (RLS) Adaptive Noise Cancellation (ANC) has proven to perform well in past research [11], [18]. The closest detector to the light source (D1) is expected to contain only maternal signals while further away detectors (D2-D5) can contain fetal information. Thus, D1 data is used as the maternal noise reference when applying ANC to D2-D5.

Fig. 9 shows the block diagram of RLS ANC applied to detector D4 as an example. The noise cancelled data from D2-D5 gives us four channels of measurement $(K=4)$ to be used in sensor fusion. The ANC filter has an order of 100, and a forgetting factor close to 1.

Fig. 9:

High Level Block Diagram of Adaptive Noise Cancellation applied to one of the detectors in TFO.

3). Computing FFTs and Power Spectrums:

The FFTs of noise cancelled four channels are computed on a rolling basis as new data becomes available. The photodetector data is downsampled to 80Hz, 60 seconds hanning window length with 30 second of overlap (50%) is used to calculate a FFT. Then the single-sided power spectrums are computed from the FFT amplitudes.

4). Thresholding:

In order to decrease the noise level and remove spurious peaks from the power spectrums, we threshold and zero-out frequency components with an amplitude <20% of the highest peak in the band of interest. The band of interest is the typical FHR range considering hypoxia (110–270BPM).

5). Outside Range Zeroing:

All peaks in the power spectrum outside the FHR range (110–270BPM) is multiplied by zero. This way we zero-out noise peaks outside the band of interest.

6). Normalization:

Since the photodetectors are placed at varying distances from the LEDs, they capture signals at different amplitude ranges. In order to correct for this difference, the processed power spectrums from four photodetectors are normalized. We normalize the power spectrums to have an area of 1 (integral from 0Hz to $f_s/2$ (40Hz) equals 1).

Fig. 6 shows an example of raw power spectrums over time (spectrograms), during one experiment, for all four $(D2-D5)$ photodetectors. Fig 7 shows the processed spectrograms after applying the processing steps outlined above. We can see that thanks to RLS ANC, MHR second harmonic, which falls into band of interest, has diminished. Additionally, thresholding the band of interest significantly reduced the background noise. Outside range zeroing completely removed low and high frequency noise outside FHR range. And finally, normalization brought all four photodetector measurements to a similar level.

Fig. 6:

Raw Spectrograms of Experiment B.

Fig. 7:

Processed Spectrograms of Experiment B.

C. Datasets

We use five datasets collected from three ewes. Experiments A and B, 48-minutes and 34-minutes long respectively, are collected from the same ewe. Experiments C and D, 37-minutes and 33-minutes long respectively, are from a second ewe. 31-minutes long Experiment E data is collected from a third ewe. The data is downsampled to 80Hz, processed and power spectrums are computed from 1-minute long windows with 50% overlap. Thus, a new power spectrum measurement becomes available every 30 seconds. KUBAI keeps estimating the FHR every 1 second using the prior distribution $p(x_t|X_{t-1}^{\left(t\right)})$, and only updates the particle weights when a measurement becomes available as shown in Fig. 4. All datasets are processed, as described in section V-B, before being input to KUBAI.

Experiment A has the cleanest data with lowest noise floor. The processed experiment A power spectrums are shown in Fig. 10. We see the presence of a strong peak in the power spectrum matching true FHR, throughout the experiment.

Fig. 10:

Processed Spectrograms of Experiment A.

Experiment B has higher noise levels, especially after minute 25, with missing FHR peaks. The processed power spectrums of experiment B are shown in Fig. 7.

Experiment C is noisier than the first two datasets. The processed power spectrums of experiment C are shown in Fig. 11. The most challenging part of this dataset is that D2 did not capture any fetal signal. This is a situation which is expected since fetal information presence is proportional to source-detector distance on the optode. With D2 being closer to the light source, it was not able to capture light that travelled deep enough to interrogate fetal layers.

Fig. 11:

Processed Spectrograms of Experiment C.

Experiment D presents the challenge that detector D2 did not capture any signal due to hardware issues. This sensor failure highlights the importance of multi-sensor fusion. Fig. 12 shows the processed power spectrums of experiment D.

Fig. 12:

Processed Spectrograms of Experiment D.

Experiment E has fairly clean FHR peaks in all channels, as seen in Fig. 13. But, we are presented with the challenge of having several crossovers between FHR, MHR and MRR harmonics that remained after ANC. We do not want KUBAI to track a maternal harmonic after the crossover point.

Fig. 13:

Processed Spectrograms of Experiment E.

D. Error Metric to Measure Accuracy

We compute the accuracy of estimating fetal heart rate (FHR) through root-mean-square error (RMSE) metric. KUBAI algorithm’s estimated output has some randomness to it since particles are sampled from probability distributions. Therefore, when computing the accuracy, it is important to take the average error over multiple runs. We report the average RMSE over 10 runs of the algorithm in this paper. Additionally, particles are initialized randomly from a uniform distribution. Thus, until the first measurement becomes available, KUBAI’s output is completely random and only converges towards the actual FHR after particle weight correction. Therefore, we compute RMSE after 1 minute.

E. Single- vs. Multi-Sensor Fetal Heart Rate Estimation

This paper presented how particle filtering algorithm can be combined with sensor fusion to create KUBAI and generate a FHR estimate every 1 second. To benchmark how sensor fusion can improve the robustness of FHR estimation, we first compute the performance of single-sensor particle filter estimates for each detector D2 to D5. The single source particle filter does not employ any sensor fusion and (8) is used as reward function in the measurement likelihood (10).

The number of particles N for both the single-sensor particle filter and KUBAI is defined as 200. When defining the number of particles the following trade-offs were considered: (1) Setting the number of particles too low will result in high variance in estimated FHR state output, causing error fluctuations, (2) Setting the number of particles too high will result in very low variance and almost constant FHR estimate over time, which is not realistic.

At the start of an experiment (time $\mathrm{t}=0$), we initialize the single-sensor and multi-sensor algorithms assuming no measurement has been taken yet. We know that the fetal heart rate (FHR) is situated between 110BPM - 270 BPM. Therefore, the initial 200 particles are uniformly sampled from the interval [110BPM, 270 BPM]. This uniform distribution represents the initial particle distribution $X_1\sim p\left(x_1\right)$.

The RMSE in BPM is computed using reference FHR values recorded through hemodynamics during the animal experiments. Table I compares the error of single-sensor particle filter estimates to sensor-fused KUBAI estimates.

TABLE I:

Fetal Heart Rate Estimation Performance Summary Single-Sensor Particle Filter vs. KUBAI with Multi-Sensor Fusion vs. Prior Work [11]

RMSE (beats-per-minute)
Experiment	Data Count	D2	D3	D4	D5	KUBAI	Prior Work
A	2821	3.7	1.2	1.4	1.5	1.3	1.2
B	1981	40.4	5.8	23.3	10.3	5.9	41.7
C	2161	24.0	43.8	3.8	4.4	5.1	31.3
D	1921	-	34.9	3.7	2.5	3.3	18.4
E	1801	9.1	3.9	2.4	2.4	2.5	5.8
Overall RMSE		23.1	24.8	10.4	5.1	3.9	24.2

We notice that there is no one single detector that always has the lowest RMSE in all five experiments. For example, for experiments A and B detector D3 has lowest RMSE while in experiment C, D4 has the lowest RMSE, and in experiments D and E D5 has the lowest RMSE for estimating FHR. Furthermore, in experiment D, detector D2 did not capture any data due to hardware issues and cannot be used for FHR tracking. There is at least one experiment where each detector stops to be accurate. This showcases the importance of multi-sensor design choice for increasing robustness against intra-, inter-experiment variability and data losses.

It is important to look at overall RMSE when comparing single-sensor vs. KUBAI’s FHR estimation performance, rather than looking at per experiment error. The goal of sensor-fusion in KUBAI is not to necessarily perform better than a single-sensor if no FHR loss happened in that sensor. However, the most accurate single-sensor changes from experiment to experiment. Thanks to sensor fusion, we can input all sensor measurements to KUBAI, whether they are good or bad. Thus, we do not need to track the quality of measurement per sensor, to select a single-sensor to rely on, which would have been necessary for single-sensor estimations without fusion. As a result of sensor fusion in KUBAI, we bring robustness to tracking of the dynamical system under observation.

When comparing the overall error across five experiments, reported in Table I, we see that KUBAI has at least 23.5% lower RMSE than D5, the best single-sensor with lowest overall RMSE. Furthermore, KUBAI’s FHR estimates have at best 84% lower RMSE than D3, the worst performing single-sensor with highest overall RMSE. On average, KUBAI’s overall RMSE is 11.95BPM (SD = 9.62BPM) lower than four single-sensor particle filter estimates, across five experiments. These results clearly demonstrate the need for sensor fusion to increase accuracy and robustness of FHR estimation.

F. Comparison with Prior Work

Previous research made use of multi-detector estimation and peak detection algorithms to track the FHR over time and reported improvement against single-source estimation performance [11]. An overview of the published FHR estimation method is presented in Fig. 14.

Fig. 14:

Prior Work on Multi-Detector Fetal Heart Rate Estimation via Peak Detection in the Power Spectrum [11].

This prior work uses bandpass filtering followed by recursive-least squares (RLS) ANC to filter high and low frequency noise, and maternal noise components. After fetal signal extraction, the power spectral densities (PSD) of the filtered signals from four detectors are calculated. The PSDs are passed through a peak detection block in which the frequency with highest power density within typical FHR range (110 – 270BPM) is reported. At the output of this peak detection block, we are left with four FHR estimates and a weight is assigned per estimate based on the credibility of each detector. The weights are set to be proportional to source-detector distance because of how light interacts with tissue. A larger source-detector distance implies capturing light that travelled deeper and thus contains more fetal information. But this comes at the expense of capturing an overall weaker signal [12]. After assigning the weights to each detector’s FHR estimate, outliers are rejected and the weighted mean FHR is outputted as the final FHR estimate. As a result, the paper reported 19% improvement in root-mean-square error (RMSE) compared to most reliable single-detector FHR estimates [11].

We evaluate and compare the performance of tracking FHR via the proposed KUBAI algorithm vs. the previously published peak detection algorithm [11]. Both methods make use of data fusion at different levels. KUBAI algorithm fuses processed power spectrum data when finding the likelihood of a measurement $p(y_{\mathrm{t}}|X_t^{\left(i\right)},Z_t)$ as described in (10) and (9). Peak detection algorithm presented in [11], fuses the FHR estimate of multiple sensors at decision level using a weighted-sums and outlier rejection approach. Since the peak detection method only relies on available measurements, it outputs a FHR estimate only every 30 seconds. In order to make a fair comparison of two algorithms’ accuracies, we assume FHR estimate of peak detection algorithm remains constant, until a new measurement is available. Everything else about the peak detection algorithm, including the weights per detector ($\mathrm{w}1=1,\mathrm{w}2=3,\mathrm{w}4=\mathrm{w}5=2$ in Fig. 14), remain unchanged.

Table I summarizes the root-mean-square error (RMSE) of estimating FHR with KUBAI and with presented prior work, computed for five different datasets. When comparing the overall performance across all five datasets, we see that KUBAI outperforms the prior work by 84% lower RMSE.

KUBAI’s performance improvement against prior work is also clearly visible when looking at scatter plots of estimate vs. reference FHR and the Pearson’s $R^2$ correlation. Fig. 15a and Fig. 15b show the scatter plots of the two algorithms. For plotting Fig. 15b, we use KUBAI’s FHR estimates from a single run of the algorithm. We see that KUBAI’s output has a very high correlation $\left(R^2=0.99\right)$ with reference FHR values while prior work only has moderate correlation $\left(R^2=\right.0.35)$ with reference. The degraded correlation of prior work is mostly due to underestimating FHR valuew above 180BPM. This can be explained by looking into FHR output and error over time for each experiment separately.

Fig. 15:

Scatter Plot of (a) Prior work vs. Reference FHR Values, (b) KUBAI vs. Reference FHR Values. The KUBAI estimates are from one run of the algorithm.

As mentioned previously, experiment A data is very clean. Thus, both KUBAI and prior work have a high FHR estimation accuracy with <2BPM RMSE. Fig. 16 shows the estimates for experiment A from prior work and one run of KUBAI. We see that after the first measurement is input at minute 1, KUBAI’s output quickly converges towards true FHR. The slightly lower RMSE of prior work over KUBAI is negligible in this clean dataset, as both have a very accurate performance.

Fig. 16:

Experiment A FHR Estimates.

When faced with a more noisier dataset, we start to see the robustness of proposed KUBAI algorithm over prior work. Fig. 17, Fig. 18 and Fig. 19 show the FHR estimates of prior work and one run of KUBAI algorithm for experiments B, C and D respectively. We see that in all these three experiments, the prior work using peak detection algorithm outputs abrupt jumps to 110BPM or 260BPM which are the bounds of FHR search span. These jumps are caused by motion in the dataset. In experiment E results, plotted in Fig. 20, we see that high error in prior work is due to outputting MHR 2nd harmonic at minute 28 as FHR. This is due to ANC not completely cancelling the maternal harmonics and in turn resulting in peak detection algorithm to output MHR harmonics with higher amplitude than FHR as it’s estimate. All these cause the prior work to not correctly track the FHR increase above 180BPM, which was observed in the scatter plot in Fig. 15a.

Fig. 17:

Experiment B FHR Estimates.

Fig. 18:

Experiment C FHR Estimates.

Fig. 19:

Experiment D FHR Estimates.

Fig. 20:

Experiment E FHR Estimates.

These results illustrate that KUBAI algorithm using multi-sensor fusion combined with particle filtering offers a more robust and more accurate FHR estimation compared to previously published methods especially when single detector FHR measurements are corrupted with motion noise or strong maternal signal contributions. The presented method is able to output accurate estimates even when a sensor does not contain any information on the desired signal (e.g. D2 in experiment C and D), by compensating for this missing information through multi-sensor data fusion. Finally, by defining a prior distribution based on existing knowledge on how the monitored physiological signal could evolve over time, the proposed algorithm can estimate the state of the monitored signal at a higher rate than measurement frequency. Prior distribution also increases robustness of KUBAI against motion by preventing the algorithm from outputting abrupt changes in FHR.

VI. Conclusion

In this paper, we presented KUBAI, a particle filtering algorithm with multi-sensor data fusion technique, for increasing the accuracy and robustness of physiological monitoring. The presented method’s performance is evaluated on Transabdominal Fetal Pulse Oximetry (TFO) device and data acquired through gold standard animal model. The performance is defined as the accuracy of estimating fetal heart rate (FHR) through non-invasively acquired multi-sensor PPG data. We first demonstrated the improvement in robustness and overall accuracy thanks to sensor fusion in KUBAI algrithm compared to single-sensor particle filter estimates, in five different datasets. Furthermore, we compared the FHR estimation RMSE and $R^2$ correlation of KUBAI to previously published methods and showed significant improvement.