A Novel Personalized Motion and Noise Artifact (MNA) Detection Method for Smartphone Photoplethysmograph (PPG) Signals

Abstract

Photoplethysmography (PPG) is a technique to detect blood volume changes in an optical way. Representative PPG applications are the measurements of oxygen saturation, heart rate, and respiratory rate. However, PPG signals are sensitive to motion and noise artifacts (MNAs) especially when they are obtained from smartphone cameras. Moreover, PPG signals are different among users and each individual’s PPG signal has a unique characteristic. Hence, an effective MNA detection and reduction method for smartphone PPG signals, which adapts itself to each user in a personalized way, is highly demanded. Here, a concept of the probabilistic neural network (PNN) is introduced to be used with the proposed extracted parameters. The signal amplitude, standard deviation of peak to peak time intervals and amplitudes, along with the mean of moving standard deviation, signal slope changes, and the optimal autoregressive (AR) model order are proposed for effective MNA detection. Accordingly, the performance of the proposed personalized algorithm is compared with conventional MNA detection algorithms. As performance metrics, we considered accuracy, sensitivity, and specificity. The results show that the overall performance of the personalized MNA detection is enhanced compared to the generalized algorithm. The average values of the accuracy, sensitivity and specificity of the personalized one are 98.07%, 92.6%, and 99.78%, respectively, while these are 89.92%, 84.21%, and 93.63% for the general one.

I. Introduction

Heart rate or heart rhythm information as one of the major physiological information can be detected from Electrocardiogram (ECG) or Photoplethysmogram (PPG) signals. There exist several types of portable ECG or PPG devices to detect heart rate. A representative portable ECG device is AliveCor ECG device (KardiaMobile) [1] which can detect ECG signal with the help of their gadget and show the ECG signal in the smartphone. However, the AliveCor requires an additional device in addition to the smartphone to obtain heart rate or heart rhythm information. Moreover, the smartwatch device can obtain PPG signals from the wrist. Still, the wrist is much weaker at MNA compared to fingertip or earlobe. The PPG signal can be measured from the fingertip by means of smartphone cameras [2, 3]. Since the penetration rate of the smartwatch is lower than the smartphone we adopted a smartphone PPG device to detect heart rates or arrhythmia. Moreover, the smartphone PPG device does not require an additional device in contrast to portable ECG device, and the smartphone device extract signals from fingertip which is more resilient at MNA compared to extracting signals from wrist by smartwatch devices.

There are various factors that distort PPG signals, which negatively affect reliable monitoring of this physiological information. Motion and noise artifact (MNA) is one of these major factors which lowers the measurement reliability by inducing unwanted variations in the measured PPG signals [4]. PPG signals obtained from smartphone cameras [2, 3] are more vulnerable to the MNAs compared to normal PPG signals. The University of Massachusetts Medical Center (UMMC) found that the subjects sometimes have tremor, in the procedure of acquiring smartphone PPG signals, due to their ages or their diseases. Moreover, subjects are found to make a mistake when using a smartphone, e.g., pressing the lens too hard or placing their fingertip in a wrong position compared to a normal pulse oximeters in which the fingertip is stably covered by a cap in a fixed position during PPG signal measurement [5]. As a result, the smartphone PPG signals corrupted by MNAs are frequently determined to be atrial fibrillation (AF) (false positive) even though the smartphone signals are acquired from healthy subjects [6-9]. Specifically, major MNAs occurred during the acquisition procedure of smartphone PPG signals include 1) hand movement, 2) fingertip misplacement, and 3)lens-pressing MNAs. The hand movement MNAs can be introduced into smartphone signals by involuntary hand movement coming from physical body movement, tremors or after-exercise. On the other hand, fingertip misplacement MNA can occur by placing the fingertip partially on the camera lens, which can measure only some parts of a fingertip with background [10, 11]. Finally, lens-pressing MNAs can occur by pressing lens harder than required.

MNA detection and removal methods for PPG signal have been proposed. Hardware-based MNA detection and removal approaches applying accelerometers to provide a reference signal to cancel motion artifacts [12-15] have been studied. The other hardware methods use additional sensors to measure only MNAs [5]. However, the hardware-based MNA reduction methods may yield frequently false positives in detecting MNAs due to its sensitiveness, e.g., PPG signals are not corrupted even though accelerometers detect motions. Further-more, hardware-based methods may require additional sensors to be attached to smartphones, which is inconvenient for users. Algorithm-based MNA reduction methods such as time-domain, frequency-domain, time-frequency techniques, and blind source separation-based techniques have been proposed [9], [16-28]. Changes in the morphology form of the signal such as amplitude, heart rate variability, and statistical properties of them are considered as time-domain analysis [9, 22, 25, 28]. Frequency domain-based methods use the frequency characteristics of signals. For example, a study based on the Hjorth parameters [29, 30] provides that the central frequency (H1) and half of the bandwidth (H2) parameters of MNAs are significantly different from those of clean signals [31]. Since MNAs in PPG signals can be highly time-varying, a mixture of time and frequency domain-based approaches such as smoothed pseudo Wigner-Ville distribution and wavelet transforms are shown to be effective in detecting MNAs in the PPG signals [20, 21, 23, 24, 26]. In other studies, blind source separation techniques such as principle component analysis (PCA) [19], and independent component analysis (ICA) [17] has been proposed to detect MNAs. Some of these algorithms could be applied to the smartphone PPG signals [32].

For smartphone PPG signals, there have been approaches of detecting MNAs considering the realistic MNA scenarios, e.g., hand movement or fingertip misplacement MNAs [7-9]. However, the limitation of these MNA detection algorithms is that the difference between each person’s characteristics is not considered. That is, the conventional approaches are mostly generalized but not personalized. For example, the PPG signals in [7-9, 24, 25, 33] are aggregated with just labeling it clean or corrupted but without labeling the subject’s identity. However, especially for clean signals, neglecting each person’s different signal characteristic can lower the accuracy in classifying the clean segments.

Personalized classifiers have been proposed to overcome the limitation of the generalized approaches in designing classifiers [34-41]. In recognizing gestures, for instances, personalized classifiers were shown to give better accuracy than generalized ones [34]. Moreover, there have been personalized approaches for smartphone-based applications [35-41]. For example, fall detection methods using smartphones were proposed based on a concept of the personalized classifier [35-37], which yielded better accuracy than generalized ones. Personalized behavior pattern recognition and unusual event detection for mobile users were proposed in [38], and the effect of personalization for the smartphone-based activity recognition was developed in [36]. The PPG signals were utilized as a unique and personalized individual’s biometric signal in [39-41]. These studies have shown that personalized classification methods can improve the performance compared to generalized ones. However, to the authors’ knowledge, there has not been proposed personalized MNA detection methods for smartphone PPG signals.

In this paper, a personalized MNA detection method for smartphone PPG signals, which detects MNAs in the smart-phone PPG signals based on concepts of signal quality index (SQI) and neural network (NN), is proposed. The proposed personalized MNA detection method is designed to classify the following three MNAs that can be introduced in acquiring smartphone PPG signals: 1) hand movement, 2) fingertip misplacement, and 3)lens-pressing MNAs. As SQI parameters for MNA detection in smartphone PPG signals, we consider six parameters in this paper: average value of signal amplitude, standard deviation of the peak-to-peak time interval (STD-T), standard deviation of successive peak values (STD-PVL), mean of moving standard deviation (E-MSD), signal slope changes, and Autoregressive (AR) model order [9, 25, 42]. These parameters are used to classify hand movement, fingertip misplacement and lens-pressing MNAs. Especially, to detect hand movement MNAs, the proposed method takes the parameter values as input of the probabilistic neural network (PNN) in a personalized way. Specifically, in this paper, the neural network is trained with the corrupted segments from all the subjects and clean segments from the target subject only. Then, the trained neural network is used to classify test smartphone PPG segments into corrupted and normal ones. Here, we evaluated the proposed method on 40 subjects. Accordingly, the performance of this personalized classifier is derived for each of the subjects in terms of accuracy, sensitivity, and specificity which is then compared with the conventional method. Since the neural network is personalized in the training phase, the trained neural network detects clean segments from PPG segments in a personalized way, which increases the detection performance.

The rest of this paper is organized as follows: Section II describes data collection and preprocessing procedure. Our proposed parameters and personalized MNA detection method are outlined in Section III. Generalized and personalized approaches in our proposed methods are described in Section III. In Section IV, the proposed personalized algorithm is evaluated by comparing it to conventional methods with recruited subjects. Moreover, the decision boundary of the method and the effect of noise on the detection probability are discussed in Section IV. Finally, Section V concludes this paper.

II. Materials and Preprocessing

A. Experimental Protocol

Smartphone video data were acquired from each subject using iPhone 4/5/6 and android phones. Forty subjects were recruited and smartphone video data from the subjects were measured following the Texas Tech University (TTU) Institutional Review Board (IRB) (IRB#: IRB2016-764) and University of Massachusetts Medical Center (UMMC) IRB (IRB #: H-14490). The thirty-five subjects were healthy, and five subjects have AF. According to the IRBs, subjects were asked to hold a smartphone by their right/left hand with covering the smartphone camera lens and flashlight by their fingertip during the measurement. The total measurement time was 90-120 seconds. As mentioned, we consider hand movement, fingertip misplacement and lens-pressing MNAs in this paper. To generate the hand movement MNA in smartphone PPG recording, thirty subjects were asked to move their hand in any arbitrary direction. Five subjects were asked to perform an act of the fingertip misplacement MNA in one set of measurement, and lens-pressing MNAs in the other set of measurement. The fingertip misplacement MNA is generated by covering the camera lens partially while the lens-pressing MNA is generated by pressing the camera lens harder than required. These MNAs are introduced in smartphone signals for 30 seconds during the acquisition procedure. Finally, the AF subjects were asked to hold a smartphone still with covering the smartphone camera properly during the whole duration of the measurement. Figs. 1a-d show these three exemplary MNA-corrupted smartphone signals, along with the AF signal respectively.

B. Preprocessing

The flowchart of the preprocessing, classifying different forms of MNA based on the proposed parameters are shown in Fig. 2. First, only the color intensity values of the green band are obtained from the recorded RGB video data. The intensity values in each frame were averaged to obtain the PPG value.

Fig. 2.

The flowchart of preprocessing and parameters selection. The preprocessing consists calculation of average value of amplitude, the high pass filter block for all the five parameters used for hand movement MNA detection, and an additional block of DWT for the AR model order. All the parameters are: average value of signal amplitude, standard deviation of peak- to-peak time interval (STD-T) [25], standard deviation of successive peak values (STD-PVL) [25], mean of moving standard deviation (E-MSD) [42], slope ratio, and optimal AR model order.

The sampling rate of the iPhone were 30 Hz while the sampling rate for android phones were in a range of 25-30 Hz. To standardize all the recordings before deriving the parameters, the recorded data from the android phones were up-sampled to the sampling rate of 30 Hz. For detecting fingertip misplacement and lens-pressing MNAs, the amplitude of the raw signal is used. On the other hand, to detect hand movement MNA, the parameter values of STD-T, STD-PVL, E-MSD, signal slope changes are calculated after high pass filter while AR calculated after the high pass filter and DWT operations. Specifically, the high pass filter had cutoff frequency of 0.5 Hz. The filtered signal is used for calculation of all parameters’ values for hand movement MNAs detection. The DWT was applied to non-stationary signals to effectively get AR parameters [43-45]. In this paper, as shown in Fig. 2, the high pass filter output is processed by discrete wavelet transform (DWT) for the AR model parameter calculation since AR parameter requires more detailed time-varying frequency information, the DWT is additionally applied only to get the AR parameter more effectively [43, 45]. Fig. 3a illustrates the steps of decomposing a signal using DWT. At each step, the wavelet function selects detail component (D) from the signal whereas scaling function selects the approximation component (A). In this procedure, the Daubechies 4 wavelet function which is shown to provide better performance in PPG denoising compared to other wavelets [46, 47] is used. First detail coefficient D1 which is the output of a high pass filter is used in calculating the AR model order, since the frequency of clean signals are usually in lower band compared to that of motion-corrupted signals [48, 49], as shown in Fig. 3b and Fig. 3c.

Fig. 3.

Decomposition of signal using DWT. a) Diagram of DWT decom- position steps. b) PPG signal, and c) the first detail coefficient (D₁) of DWT.The segments with higher amplitude values, and aperiodic waveform are thecorrupted part and the other segments are the clean parts.

III. Methods

The flowchart of our hand movement MNA detection method with proposed parameters is shown in Fig. 4. The proposed parameters considered for effective MNA detection are described in Section A. The standardization and normalization of the parameters are described in Section B. Section C explains about the proposed personalized MNA detection.

Fig. 4.

The overall flowchart of processing the hand movement MNA- related parameters for the personalized/generalized classifier. All the proposed parameters are given to a standardization and normalization block prior of being used as the probabilistic neural network input layers.

A. Parameters From Smartphone PPG Signals

Fig. 3 shows that MNA-corrupted signals have different peak-to-peak intervals, amplitude, and morphology compared to clean ones. Moreover, adjacent pulses are observed to be irregular and aperiodic in corrupted signals whereas they are regular and periodic in clean ones. Specifically, the fingertip misplacement and lens-pressing MNA-corrupted smartphone PPG signals have different signal amplitude values compared to clean ones. Conversely, hand movement-corrupted PPG signals have different standard deviation of peak-to-peak time interval, standard deviation of successive peak values, mean of moving standard deviation, signal slope changes, and autoregressive model. Each parameter for the hand movement MNA is determined based on fragment-by-fragment calculation. The amplitude of raw signal is straightforward. The definition of each parameter is described in subsections.1-6, respectively.

1). Average Value Of Amplitude (AMP_mean):

Denoting by AMP_mean the average value of signal amplitude, AMP_mean is calculated as

$$AM{P}_{\text{mean}}=\frac{1}{K}{\displaystyle \sum _{i=1}^K\left(Am{p}_k\right)}$$ (1)

where K is the number of samples in a total signal and Amp_k is the amplitude value of the total signal at the i^th peak point. MNA induced by fingertip misplacement and lenspressing will give significantly higher amplitude values than the amplitude value of clean signal. Hence, it is expected that MNA signal has higher values than AMP_mean.

2). Standard Deviation of Peak-To-Peak Time Interval (STD-T):

The STD-T value of the n^th fragment is the standard deviation of the time interval difference between peaks and calculated by:

$$STD-T_n=\sqrt{\frac{1}{M}{\displaystyle \sum _{m=1}^M{\left(T_{n,m}-T_{n,m-1}\right)}^2}}$$ (2)

where T_n,m is the time value at the m^th successive peak of the n^th fragment and M is the number of peaks in each fragment. Since the time interval between peaks is periodic for clean parts and aperiodic for MNA signal, it is expected that STD-T for MNA fragment to be larger than the clean one.

3). Standard Deviation Of Successive Peak Values (STD-PVL):

The STD-PVL of the n^th fragment with M peaks is calculated by:

$$STD-PV{L}_n=\sqrt{\frac{1}{M}{\displaystyle \sum _{m=1}^M{\left(PV{L}_{n,m}-PV{L}_{n,m-1}\right)}^2}},$$ (3)

where PVL_n,m is the successive peak value at the m^th peak of the n^th fragment. The clean signal has similar and regular values of peaks while the MNA signal has diverse values of successive peaks. Here the STD-PVL is expected to be smaller in clean segments than that of corrupted signals.

4. ) Mean of Moving Standard Deviation (E-MSD):

The E-MSD in each fragment is calculated by:

$$E-MS{D}_n=E\left[MS{D}_{n,m,W}\right],$$ (4)

where E[x] is the mean of x and the MSD of the n^th fragment and at the m^th sample point is derived by

$$MS{D}_{n,m,W}=\sqrt{\frac{{\displaystyle \sum _{i=m-W+1}^m{\left(y_i-\overline{y}\right)}^2}}{W-1}},\forall m=1,\dots ,N_{\text{samp}},N_{\text{samp}}=16$$ (5)

where y_i is i^th sample value in the sliding window (D_msd) of length W, ӯ is the average value of samples in the sliding window, and N_samp is the number of samples in each fragment from the down sampled original signal. The E-MSD in the MNA fragments supposed to be higher than the clean segments since the samples are not distributed even from the point view of average.

5). Slope Ratio (SR):

The SR [9] in each fragment is derived by:

$$S{R}_n=\left|\frac{S{P}_n}{S{N}_n}\right|,$$ (6)

where SP_n is the maximum value of positive slopes and SN_n is the minimum values of negative slopes of n^thfragment which are given by:

$$S{P}_n=\max \left(\left(I_{n,m}-I_{n,m-1}\right)>0\right),m=1,\dots ,S$$ (7)

$$S{N}_n=\min \left(\left(I_{n,m}-I_{n,m-1}\right)<0\right),m=1,\dots ,S$$ (8)

where I_n,_m is the peak-to -peak interval of the m^th sample point in the n^th fragment and S is the number of samples in each fragment. The slope between samples in clean signal is identical which results to change in a specific threshold value. However, the slope changes in MNA fragments are antithetical compared to clean ones resulting to a SR value over the threshold.

6). Optimal AR Model Order $\left(A{R}_p^{*}\right)$ :

The ARp* value is derived from the first detail component of DWT signal and determined from the below equation so that the Akaike information criterion (AIC) [50] value is minimized:

$$Arg{}_p(AIC)=-2\ln (L)+p,$$ (9)

where L denotes the likelihood function and p represents the AR model order given by

$$x_t={\displaystyle \sum _{i=1}^p{\varphi }_i}{x}_{t-i}+{\epsilon }_t,$$ (10)

where x_t is the current value of sample at pointt, ɛ_t the prediction-error term, and ϕ_i represents the AR coefficients at the i^th instant and estimated using the Yule-Walker equation [51, 52]. The ARp* is observed to be effective in quantifying the degree of motion corruption which is larger for the corrupted signal than the clean ones [53-55]. AIC, Bayes information criterion (BIC) [56], minimum description length (MDL) criterion [57] and Kullback information criterion (KIC) [58] are widely used to determine the ARp*. In this paper we adopted AIC.

B. Standardization AND Normalization of Parameters

The parameters used as input values in the neural network are required to have the same range of maximum and minimum values to have uniform effect on the network. Here, the maximum and minimum values of each extracted parameter is different. Therefore, to make them have uniform impact on the classifier and increase the performance of our proposed neural network, the input values are standardized and normalized [59, 60]. The values of each parameter are standardized by:

$$x_{qp\text{standard}}=\frac{x_{qp}-\overline{x_p}}{{\sigma }_{x_p}},$$ (11)

where x_qp is the value of q^th point of the p^th parameter, ${\stackrel{-}{x}}_p$ is the mean value of p^th parameter and ${\sigma }_{x_p}$ is the standard deviation of the p^th parameter The normalized data is derived by applying a hyperbolic tangent sigmoid function on the standard data as follows:

$$x_{qp\text{normal}}=\text{tansig}\left(x_{qp\text{standard}}\right),$$ (12)

This function rescales all vector components of a parameter to the range of [−1, +1].

C. MNA Detection Using the Probabilistic Neural Network

1). Neural Networks:

A neural network is a mathematical model usually used in complicated classification problems. neural networks have widely been used in designing classifiers since it can generate a decision boundary in complicated settings without full knowledge of the statistical properties of the observed data [59, 61, 62]. there have been studies which applied a concept of neural network to the classification of ecg signals e.g. normal, atrial fibrillation, pattern recognition, and noise reduction [63-68]. recently, neural network was also used in a personalized setting for ecg signals classification [69].

The neural network consists of computational units called nodes as shown in Fig. 5. A node applies a function to the weighted sum of its inputs. The nodes are arranged in series of multiple connected layers known as input layers, hidden layers and output layers. In this paper, a Probabilistic Neural Network (PNN) [70] which is a feed-forward type of neural networks is adopted with two output classes as shown in Fig. 5. The first layer known as input layer is a distribution layer. The number of nodes N is equivalent to the number of samples. Here, the hidden layer consists of pattern layer and summation layer. In the pattern layer, the Gaussian function is used as a kernel analysis to compute the distance from the training input which is labelled as clean or corrupted (clean: class 1, corrupted: class 2). Since Gaussian function is reasonably assumed in the classification problem due to its lower false acceptance rate, we adopted Gaussian function in the hidden layer of PNN [59,71,72]. The distance of each sample from the training input is derived using the following equation [70]:

$${\varnothing }_i(x)=\frac{1}{{\sigma }^d{(2\pi )}^{d{/}_2}}\frac{1}{m}{\displaystyle \sum _{j=1}^m{e}^{-\frac{{\left(x-x_i\right)}^T\left(x-x_i\right)}{2{\sigma }^2}}},$$ (13)

where, j is the number of the input parameter, m is the total number of training inputs, X_i is the training input vector, d is the dimension of the measurement vector space which is 5 in our experiment, and σ μ is the variance which controls the amount of smoothness. Then, the sums of these contributions are calculated for each class of input to produce output probabilities. From the calculated probabilities, compete transfer function at the summation layer picks the maximum of probabilities and produces 0 for clean (class 1) and 1 for MNA (class 2). A diagram of the training and testing phase of the proposed personalized MNA detection algorithm along with the generalized algorithm is shown in Fig. 6. The training set of the proposed personalized MNA classifier for subject i consists of the clean data from this selected subject and MNA data from all the other 29 subjects (since the AF subjects do not have any MNA data). The testing set of the personalized classifier is only the data from the clean and MNA part of the selected subject i. In the personalized algorithm, the train data is developed for each user based on the clean smartphone PPG characteristic of that individual while it has the MNA data from the other subjects as well. However, the conventional generalized MNA detection method trains the neural network by all the clean and MNA data of all available subjects except the selected subject, without considering the unique characteristic of each subject.

Fig. 5.

Architecture of probabilistic neural network. The probabilistic neural network consists of input layer, pattern layer, summation layer, and output layer. Class 1 are the segments classified as clean and class 2 are the segments classified as the MNA by the proposed neural network.

Fig. 6.

Block diagram of personalized and generalized training and testing stage.

This generalized network is then tested on the data from the selected subject. The training and testing algorithm of the personalized algorithm is repeated for each subject and evaluated with the generalized approach for that individual. The number of training samples is 1094 segments for the personalized and 3990 segments for the generalized algorithm; while the number of test segments is 114 for both the personalized and generalized algorithms in each iteration.

IV. Results

Our considering parameters’ values are listed in Table 1. The values of L_param, D_shift, D_MSD are reached by grid search method. The PPG signals recorded by smartphone camera are divided into fragments with the length of 150 samples (L_param = 150). For each subject, the parameter values for hand movement MNA described in section III are calculated in each fragment. The fragments are shifted with the length of (D_shift = 30 samples) each time to get parameter values. PPG recordings are labeled by clean or corrupted for each segment of L_decision = 7s. For deriving the values of the E-MSD a sliding window (D_msd) whose length is 3 samples is used. D_msd is optimally derived by a grid search with changing the value of the D_msd from 3 to 7 samples.

TABLE I.

PERFORMANCE COMPARISON BETWEEN GENERALIZED ALGORITHM AND H1 AND H2.

Method	Accuracy	Sensitivity	Specificity
Proposed Generalized Algorithm	89.92%	84.21%	93.63%
HI	73.36%	45.55%	86.06 %
H2	64.82%	34.10%	78.46%

The parameter values of the clean and corrupted fragments are described in section A. The results of lens-pressing and fingertip misplacement MNA are presented in section B. The hand movement MNA detection algorithm using probabilistic neural network is compared with the conventional H1 and H2 algorithms and described in section C. The performance of our proposed neural network-based hand movement MNA detection algorithm is evaluated by comparing it with conventional MNA detection algorithm and presented in section D.

A. Evaluation of Selected Parameters

The signal amplitudes of clean (C) and motion corrupted (M) PPG signals from 40 subjects are shown in Fig. 7. From Fig. 7, it is observed that the smartphone PPG signals corrupted by motions are shown to be significantly different from the clean smartphone PPG signals in terms of pulse amplitude. Moreover, the clean signals have different statistic characteristic between different subjects as shown in Fig. 7, which supports the contribution of personalized classifier.

Fig. 7.

Normalized amplitude variation of clean (C) and MNA (M) PPG signals of 40 subjects. The boxplots from number 1 to 30 are related to hand movement MNA; the boxplots from number 31 to 35 are related to AF subjects; the boxplots from number 36 to 40 are related to fingertip misplacement introduced MNA; the boxplots from number 41 to 45 are related to lens-pressing induced MNA by the same subjects in the fingertip misplacement introduced MNA. The central line represents the median; the 25th and 75th percentiles are the bottom and top edges of the box, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the ’+’ symbol.

The values of the five selected parameters for the hand movement MNA with the PPG signal of a subject is shown in Fig. 8. The values of the parameters are derived based on the fragment basis calculation (L_param = 150 samples and D_shift = 30 samples). MNAs are modeled in smartphone PPG signals by asking subjects to move their hands for 30 seconds as mentioned in Section II, A. The acquired values of the five parameters then labeled as clean or corrupted segments ( L_decision = 7s). The start and end time of the corruption were written down as T_MNA,start and T_MNA,end. The segment is labeled as corrupted if more than 15% of a segment (more than 1s) is in the [T_MNA,start, T_MNA,_end] interval. As expected the corrupted fragments have higher parameter values than clean ones for the considered parameters as shown in Fig. 8.

Fig. 8.

(a) Smartphone PPG signal of an individual, (b) standard deviation of peak-to-peak time interval (STD-T) [25] in each fragment, (c) standard deviation of successive peak values (STD-PVL) [25] in each fragment, (d) mean of moving standard deviation (E-MSD) [42] in each fragment with the sliding window size of 3 samples, (e) slope ratio, and (f) optimal AR model order.

A statistical comparison between the extracted parameters from the clean part of signal between two individuals is shown in Fig. 9a. All five parameters are shown to be significantly different from each other. Moreover, selected parameters are significantly different between the two subjects (p< 0.05) for the clean part of the signals. In Fig. 9b, four out of five parameters of the hand movement MNA part are marked as statistically significant between the subjects. As a conclusion, Fig. 9 demonstrates significant difference in the selected parameters for the hand movement MNA and between the subjects, specifically the clean parameters, which proves that the personalized algorithm with the extracted parameters performs better than the generalized one.

Fig. 9.

Comparison between clean and MNA part of each parameters between two subjects selected randomly. (a) Clean values of all the five parameters, (b) MNA values of all the five parameters. (*) demonstrates the statistical significance difference (p< 0.05) of mean value between two subjects. p-values are derived from a two-sample t-test. The central line represents the median; the 25th and 75th percentiles are the bottom and top edges of the box, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the ‘+’ symbol.

B. Detection of Lens-Pressing and Fingertip Misplacement MNA

In our experiment, the healthy subjects were asked to induce three major categories of MNAs including 1) hand movement, 2) fingertip misplacement, and 3) lens-pressing MNAs, in the middle of the measurement. On the other hand, the AF subjects were asked to hold the smartphone still. As shown in Fig. 2, the parameter AMP_mean is used to discriminate the hand movement MNA from the fingertip misplacement MNA, lens-pressing MNA, and clean signals. If the amplitude values of the samples in the range of [Amp_T(MNA,start) Amp_T(MNA,end)] are higher than the value of the AMP_mean, then the corresponding samples in the signal are determined to be fingertip misplacement or lens-pressing MNA. However, since hand movement MNA has different characteristics compared to fingertip misplacement and lens-pressing MNA, the hand movement MNA is not detected using this method (see Section IV-D for the detection of hand movement MNA). The results show that our algorithm perform 100% accuracy in detecting the lens-pressing and fingertip misplacement MNA and discriminating these two major forms of MNA from the hand movement MNA and AF signal as well.

C. Evaluation of the Proposed Parameters With Neural Network

1). The Effect of Spread Factor:

The probability density function is adopted as a kernel function in the hidden layer of the neural network. The efficiency of the network is affected by the standard deviation in this hidden layer known as the spread factor (σ). This function computes the distance of each testing data from the training data. In Fig. 10, two sample points from the clean and corrupted data of a subject are selected. These two sample points are the cluster head of each category. Specifically, the clean data point is the average of all the clean samples while the corrupted data is derived from the average values of the corrupted samples. The probability density functions of these two-sample points with two different values of spread factor (σ = 0.1and σ = 0.9) are derived. It is shown in Fig. 10b that thex-axis and y-axis values of the probability density distribution of the STD-T parameter varies from −1.5 to 1.5. These ranges of variations are similar for the distribution function of the STD-PVL parameter. The distribution functions of these parameters start to overlay on each other at x = 0 and y = 0, which indicates that two functions have a large area in common with the choice of σ = 0.9. However, the two distribution functions have common area only for the values of 0 < y < 0.5 with the choice of σ = 0.1. Therefore, this common area is less and distinct for the value of σ = 0.1.

Fig. 10.

Probability density of clean and corrupted sample points with changing spread term (up spread term σ = 0.1, bottom μ spread term σ = 0.9). The probability density function with higher value is for corrupted sample and the lower value is for clean sample.

Hence, choosing lower values of σ results in more distinct area between the functions while choosing larger values of σ gives more common area between the two functions. As a result, it is difficult to make difference between clean and corrupted distribution for large values of σ which causes false detection and decreeing the performance. In our proposed method, the value of σ = 0.1 is derived in a sub-optimal way using grid search algorithm based on the iteration of experiments by changing the value of this parameter from the set of {0.1, 0.2, 0.3, ..., 1}. For each value of σ, the performance of the classifier is calculated and the sub-optimal value of σ= 0.1 is chosen for all the subjects in the personalized and generalized algorithms.

2). Evaluation of Parameters With Neural Network:

The performance of the proposed hand movement MNA detection methods is compared with conventional MNA detection algorithms. The H1 and H2 parameters [29, 30] are considered since it is shown that these two parameters are recognizable between the clean and corrupted PPG signals with a predefined threshold and extensively used for PPG signal analysis applications [24, 31, 73]. Here H1 represents the central frequency of the signal whereas the H2 represents the half-bandwidth of the signal and are defined as

$$H_1=\sqrt{\frac{{\overline{w}}_2(n)}{{\overline{w}}_0(n)}},H_2=\sqrt{\frac{{\overline{w}}_4(n)}{{\overline{w}}_2(n)}-\frac{{\overline{w}}_2(n)}{{\overline{w}}_0(n)}}.$$ (14)

Where ${\overline{w}}_i(n)$ is the i^th order spectral moment of the signal and derived by

$${\overline{w}}_i(n)={\displaystyle {\int }_{-\pi }^{\pi }{w}^i}{S}_x\left(e^{j\omega }\right)d\omega ,$$ (15)

that $S_x\left(e^{j\omega }\right)$ is the power spectrum of the signal in each segment.

For direct comparison, the segment length (L_decision = 7s) is applied for evaluating the method.

The performance of our proposed algorithm is evaluated in terms of accuracy, sensitivity and specificity. These terms are defined as:

$$\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}$$ (16)

$$\text{Sensitivity}=\frac{TP}{TP+FN}$$ (17)

$$\text{Specificity}=\frac{TN}{TN+FP}$$ (18)

where T P and T N are the number of true positive (segments labeled as MNA are detected correctly) and true negative (segments labeled as clean are detected correctly) while FP and FN are the number of false positive and false negative segments.

It is shown in Table 2 that our proposed parameters with probabilistic neural network give 89.92%, 84.21, and 93.63% for the accuracy, sensitivity and specificity on average which is higher compared to conventional H1 and H2 algorithms. The average accuracy of the H1 and H2 are 73.36% and 64.82%, and the sensitivity are 45.55% and 34.10% in average. The average specificity of the conventional H1 and H2 algorithms are 86.06% and 78.46% for all the thirty subjects.

TABLE II.

PARAMETERS FOR DATA FRAGMENT

Parameter	Description	Value
Ldecision	Segment length	3,5, and 7 seconds
Lparam	Fragment length	150 samples
Dshift	Shift length	30 samples
DMSD	sliding window length	3 samples

D. Performance of the Proposed MNA Detection Algorithm: Personalized and. Generalized

In our algorithm, five different parameters are used as input of the probabilistic neural network. The parameters are normalized and standardized before feeding to the classifier to have similar importance to the classifier [59, 60]. To observe the effect of decision boundary between the parameters, the PPG signal of one individual is selected. The boundary decision of neural network between clean and hand movement MNA signals are shown in Fig. 11 between paired parameters. The parameters are selected as pairs to be shown more clearly in two dimensions. Moreover, since the size of each input parameter is large, to visualize the decision boundary more evident, around ~30% of whole data is shown in the figure. The red and yellow area are the neural network decision area of clean and MNA signal respectively.

Fig. 11.

Neural network decision boundary with 2 set of parameters in the personalized algorithm. (a) STD-T and STD-PVL, (b) STD-PVL andE-MSD, (c) E-MSD and slope ratio, and (d) slope ratio and optimal AR order.

The accuracy, sensitivity, and specificity of the personalized and generalized algorithm with the fixed value of (σ = 0.1) are described in Table 3. Moreover, the difference between the performance of the personalized and generalized is presented along the personalized performance. The number of segments used for training the personalized classifier are 1094 and the number of segments used for test are 114. The generalized classifier is trained with 3990 segments and tested on 114 segments.

TABLE III.

Performance comparison between personalized and generalized algorithm.

	Personalized Algorithm			Generalized Algorithm
Subject	Accuracy (+ change)	Sensitivity (+ change)	Specificity (+ change)	Accuracy	Sensitivity	Specificity
1	95.61% (+2.63)	84.85% (+6.06)	100.00% (+1.23)	92.98%	78.79%	98.77%
2	94.74% (+1.75)	88.57% (+5.71)	97.47% (+0.0)	92.98%	82.86%	97.47%
3	99.12% (+0.0)	97.14% (+0.0)	100.00% (+0.0)	99.12%	97.14%	100.00%
4	96.49% (+5.26)	80.95% (+9.52)	100.00% (+4.30)	91.23%	71.43%	95.70%
5	98.25% (+13.16)	94.29% (+14.29)	100.00% (+12.66)	85.09%	80.00%	87.34%
6	100.00% (+7.02)	100.00% (+3.45)	100.00% (+8.24)	92.98%	96.55%	91.76%
7	96.49% (+4.39)	88.57% (+2.86)	100.00% (+5.06)	92.10%	85.71%	94.94%
8	98.25% (+1.75)	94.29% (+5.71)	100.00% (+0.0)	96.49%	88.57%	100.00%
9	98.25% (+1.75)	94.29% (+5.71)	100.00% (+0.0)	96.49%	88.57%	100.00%
10	98.25% (+6.14)	94.29% (+2.86)	100.00% (+7.60)	92.10%	91.43%	92.40%
11	99.12% (+1.75)	96.43% (+3.57)	100.00% (+1.16)	97.37%	92.86%	98.84%
12	98.25% (+3.51)	92.86% (−3.57)	100.00% (+5.81)	94.74%	96.43%	94.19%
13	98.25% (+11.40)	94.29% (+2.86)	100.00% (+15.19)	86.84%	91.43%	84.81%
14	96.49% (+2.63)	88.57% (+8.57)	100.00% (+0.0)	93.86%	80.00%	100.00%
15	99.12% (+0.88)	97.14% (+0.0)	100.00% (+1.27)	98.25%	97.14%	98.73%
16	100.00% (+50.0)	100.00% (+66.2)	100.00% (+0.0)	50.00%	33.72%	100.00%
17	99.12% (+4.39)	96.43% (+0.0)	100.00% (+5.81)	94.74%	96.43%	94.19%
18	96.49% (+3.51)	88.57% (+5.71)	100.00% (+2.53)	92.98%	82.86%	97.47%
19	100.00% (+8.77)	100.00% (+20.0)	100.00% (+3.80)	91.23%	80.00%	96.20%
20	98.25% (+6.14)	92.86% (+3.57)	100.00% (+6.98)	92.11%	89.29%	93.02%
21	92.98% (+5.26)	71.43% (+10.71)	100.00% (+3.49)	87.72%	60.71%	96.51%
22	96.49% (+11.40)	85.71% (+25.00)	100.00% (+6.98)	85.09%	60.71%	93.02%
23	100.00% (+8.77)	100.00% (+7.14)	100.00% (+9.30)	91.22%	92.86%	90.70%
24	92.98% (+7.89)	88.57% (+20.0)	94.94% (+2.54)	85.09%	68.58%	92.40%
25	100.00% (+3.51)	100.00% (+5.41)	100.00% (+2.60)	96.50%	94.60%	97.40%
26	99.12% (+1.75)	97.14% (+0.0)	100.00% (+2.53)	97.37%	97.14%	97.47%
27	94.74% (+4.39)	85.71% (+11.90)	100.00% (+0.0)	90.35%	73.81%	100.00%
28	96.49% (+4.39)	88.57% (+2.86)	100.00% (+5.06)	92.10%	85.71%	94.94%
29	99.12% (+4.39)	96.43% (+0.0)	100.00% (+5.81)	94.74%	96.43%	94.19%
30	100.00% (+3.51)	100.00% (+5.56)	100.00% (+2.56)	96.49%	94.44%	97.44%
31	100.00% (+5.26)	-	100.00% (+5.26)	94.73%	-	94.73%
32	100.00% (+14.04)	-	100.00% (+14.04)	85.96%	-	85.96%
33	100.00% (+33.33)	-	100.00% (+33.33)	66.66%	-	66.66%
34	100.00% (+17.54)	-	100.00% (+17.54)	82.45%	-	82.45%
35	100.00% (+22.81)	-	100.00% (+22.81)	77.19%	-	77.19%
Mean ± STD	98.07% ±2.02	92.60 ±6.54	99.78%±0.93	89.92%±9.35	84.21% ± 14.0	93.63% ±7.04

The overall performance (accuracy, sensitivity, and specificity) of the personalized method are higher than those of the generalized one except for two subjects among 35 subjects (See Table 3). For these two subjects, only sensitivity of the generalized algorithm is slightly higher (~3.5%) than that of the proposed algorithm. Moreover, the accuracy of the personalized classifier is 100% for all the AF subjects which indicates that the proposed algorithm is able to discriminate the MNA signal from the AF signal in all cases.

As shown in Table 3, for some subjects the personalized sensitivity is lower than 90% for the L_decision = 7s, We tested our personalized algorithm by changing the value of L_decisionfrom 7s to 5s and 3s to see the effect of segment size on the performance.

Fig. 12a shows that the choice of segment length of L_decision= 3s and L_decision= 5s and spread factor (σ = 0.1) can help to improve the sensitivity for those subjects with the personalized sensitivity lower than 90% while maintaining the accuracy and sensitivity of personalized method higher than generalized (see Fig. 12b and Fig. 12c). These results show that for most subjects choosing smaller value of L_decision is helpful to increase the sensitivity of algorithm when there is small interval of MNA in one segment.

Fig. 12.

Comparison between the effect of segment length on the personalized algorithm performance. Blue line and circles (○) are the performance with L_decision = 7sec, red dash and squares (□) represents the L_decision = 5sec segment size, green line and stars (*) demonstrates the performance with L_decision = 3sec, and diamond with magnet line demonstrates the performance of generalized. a) comparison of the sensitivity. Red boxes indicate the subjects with sensitivity less than 90% L_decision = 7sec, which are improved by changing the segment length. The sensitivity for the AF subjects (subjects31- 35) is not defined as number, therefore the plot shows the sensitivity of the subjects 1-30. b) comparison of accuracy. c) comparison of specificity.

E. Sensitivity Analysis With AWGN

The accuracy of the personalized algorithm is also evaluated in the presence of Gaussian noise with different SNR values. As shown in Fig. 13. With AWGN SNR of −20 dB the detection probability of the personalized algorithm is 99%. This detection probability for SNR values of −15 dB, −10dB and −5 dB are 97%, 96% and 88% respectively. The value of the detection probability will decrease to the 12% for SNR of 0 dB and goes to zero for the SNR value of 5dB.

Fig. 13.

Detection Probability of personalized algorithm by adding AWGN for −20 dB, to 5dB SNR levels to clean part of PPG signal with the step size of 5dB SNR.

V. Conclusion

PPG is useful for mobile health monitoring since it provides blood pressure, heart rate, and respiratory rate in real time. In this paper, six parameters are used for detecting the MNA. The AMP_mean is used for detecting the lens-pressing and fingertip misplacement MNAs from the hand movement MNA. The other five effective parameters are proposed for detecting hand movement MNAs in smartphone PPG signals: STD-T, STD-PVL, E-MSD, signal slope changes, and AR model order. Moreover, five AF subjects are also considered to evaluate the proposed algorithm in discriminating clean AF signal from the MNA-corrupted signals. Using these parameters as input of probabilistic neural networks, a personalized MNA detection algorithm is proposed Specifically, the personalized MNA detection algorithm is proposed since the characteristics of PPG such as heart rate, peak-to-peak amplitude, and morphology vary among individuals. The test results of our proposed method on 40 subjects have shown that our proposed parameters are shown to give significant different values between clean and corrupted PPG segments. Moreover, the parameter values among subjects are shown to be significantly different.

Our proposed MNAs classification method have shown to give 100% accuracy in discriminating the lens pressing and fingertip misplacement MNAs from the hand movement MNA. The proposed neural network-based hand movement MNA detection method have shown the better accuracy compared to conventional H1 and H2 algorithms. Specifically, the proposed generalized MNA detection algorithm provided 89.92% accuracy whereas H1 and H2-based algorithms give 73.36% and 64.82% respectively.

From the comparison results between the generalized and personalized ones, on the other hand, the personalized MNA detection algorithm has provided better accuracy, sensitivity, and specificity compared to generalized one. For example, when the L_decision is 7s, the personalized algorithm has provided accuracy, sensitivity and specificity of 98.07%, 92.60% and 99.78%, respectively while the generalized algorithm has given 89.92%, 94.21%, 93.63% respectively (on average).

The sensitivity of our personalized algorithm is tested by varying L_decision with values of 3s, 5s, and 7s. The choose of smaller segment length ( L_decision =3s, 5s) has improved the personalized sensitivity for ~50% numbers of the subjects. The results show that the choice of the decision segment length is another parameter in designing the personalized algorithm. The performance of personalized algorithm may not be the same with the choice of specified segment length. our proposed parameters with the probabilistic neural network are expected to give personalized capability with high accuracy.

Fig. 1.

Changes in the morphology of PPG signal by different examples of induced MNA and the AF. a) PPG signal from a subject with AF, b) MNA induced by lens-pressing, c) MNA induced by fingertip misplacement, and d) MNA induced by hand movement.

Biography

Fatemehsadat Tabei received her B.S. and M.S. degrees in electrical engineering from Shiraz University, Shiraz, Iran. Since 2017, she has been a Ph.D. student at Texas Tech University, USA. Her current research interests include biomedical signal processing, biomedical image processing, and machine learning.

Rajnish Kumar received his B. Tech from National Institute of Technology, Patna, India and his M.S. and PhD from Texas Tech University Lubbock TX, USA. His current area of research includes applications of machine learning techniques in drug discovery and personalized medicine.

Tra Nguyen Phan received her B.S degree in Electrical Electronics Engineering from Ho Chi Minh City University of Technology, Vietnam, in 2012 and the M.S. degree in Micro and Nano Systems Technology from Buskerud and Vestfold University College, Norway in 2015.

David D. McManus received the B.S. degree in biology from Brown University, the M.D. degree in medicine from the University of Massachusetts Medical School, and the M.Sc. degree in clinical investigation from the University of Massachusetts Medical School. He is currently a Clinical and Research Cardiologist and a Cardiac Electrophysiologist with the University of Massachusetts Medical School, an Associate Professor of medicine, and the Director of the University of Massachusetts Medical Center’s Atrial Fibrillation Treatment Program, which he created in 2009.

Jo Woon Chong received the B.S., the M.S. and Ph.D. degrees in electrical engineering from Korea Advanced Institute of Science and Technology, Daejeon, Korea, in 2002, 2004, and 2009, respectively. From 2010 to 2012, he was a postdoctoral fellow at the Massachusetts Institute of Technology (MIT). He was a research assistant professor at the worcester Polytechnic Institute, USA from 2012 to 2016. Since he has been an assistant professor at Texas Tech University, USA. His current research interests include biomedical signal processing, non-invasive physiological monitoring, sensors and wireless communication for medical care, machine learning techniques for healthcare data, and home monitoring of health and disease.