Motion and Noise Artifact-Resilient Atrial Fibrillation Detection using a Smartphone

Abstract

We have recently found that our previously-developed atrial fibrillation (AF) detection algorithm for smartphones can give false positives when subjects’ fingers or hands move, as we rely on proper finger placement over the smartphone camera to collect the signal of interest. Specifically, smartphone camera pulsatile signals that are obtained from normal sinus rhythm (NSR) subjects but are corrupted by motion and noise artifacts (MNAs) are frequently detected as AF. AF and motion-corrupted episodes have the similar characteristic that pulse-to-pulse intervals (PPIs) are irregular. We have developed an MNA-resilient smartphone-based AF detection algorithm that first discriminates and eliminates MNA-corrupted episodes in smartphone camera recordings, and then detects AF in MNA-free recordings. We found that MNA-corrupted episodes have highly-varying pulse slope, large turning point ratio, or large kurtosis values in smartphone signals compared to MNA-free AF and NSR episodes. We first use these three metrics for MNA discrimination and exclusion. Then, AF is detected in MNA-free signals using our previous algorithm. The capability to discriminate MNAs and AFs separately in smartphone signals increases the specificity of AF detection. To evaluate the performance of the proposed MNA-resilient AF algorithm, 99 subjects, including 88 study participants with AF at baseline and in NSR after electrical cardioversion as well as 11 participants with MNA-corrupted NSR, were recruited. Using iPhone 4S, 5S, and 6S models, we collected 2-minute pulsatile time series from each subject. The clinical results show that the accuracy, sensitivity and specificity of the proposed AF algorithm are 0.97, 0.98, 0.97, respectively, which are higher than those of the previous AF algorithm.

I. Introduction

Recently, use of smartphones for atrial fibrillation (AF) detection has been gaining attention for clinical applications due to its convenience and efficiency in the prediagnosis and management of patients with, or at risk for, AF, given the paroxysmal, short-lived, and frequently asymptomatic nature of this arrhythmia [1–3]. AF monitoring is important because, despite often being paroxysmal and associated with minimal or no symptoms, AF is associated with severe adverse health consequences including stroke, heart failure, and death [4]. However, motion and noise artifact (MNA) that occurs during smartphone application operation may cause false positive AF since it mimics AF characteristics by generating irregular peaks in smartphone recordings. Hence, the accuracy of the AF detection can be noticeably decreased when MNAs occur frequently during the smartphone recording, e.g., normal sinus rhythm (NSR) signals corrupted by MNAs can be misclassified as AF.

MNAs in smartphone camera recordings can be induced by (1) the misplacement of the subject’s fingertip or (2) movement of the subject’s hand. Since a smartphone camera, unlike a pulse oximeter or Holter monitor device, is not mainly designed to measure physiological signals, the signals obtained from smartphone camera recordings of a finger are not as well protected from MNA. There have been hardware [5, 6] and software-based [7–15] MNA detection/reduction approaches proposed. However, they are not directly applicable to smartphone applications since these approaches are designed primarily for pulse oximeter signals. Moreover, these approaches do not have AF detection capability. Hence, an MNA discrimination algorithm which efficiently excludes MNA episodes in smartphone signals needs to be developed and to be incorporated into AF detection algorithms.

Our previous smartphone-based AF detection application was able to detect AF, with 96% accuracy, from 120-second segments of smartphone recordings [1, 2]. This high accuracy was in part due to controls in the study that minimized MNA. However, we discovered that this previous AF detection algorithm misclassifies NSR signals as AF if there are more than twelve MNA-corrupted beats present among sixty beats. Another limit of this algorithm is that it cannot be applied to a variety of lengths of smartphone recordings because a decision boundary distinguishing NSR from AF was derived optimally only for 120-second length signals. Thus, to further optimize the prior work, an enhanced and adaptive AF algorithm for smartphone signals was sought. The authors are not aware of any studies providing a smartphone-based AF detection method that is resilient to MNA and automatically sets its AF decision boundary adaptively to different lengths of recordings.

In this paper, we propose an MNA-tolerant AF detection algorithm for iPhone 4S/5S/6S, which first discriminates MNA-corrupted from MNA-free (or clean) beats, and then detects AF beats from only the clean beats as shown in Fig. 1. To achieve this MNA-resilient AF detection algorithm, we introduce time-domain parameters that are used to quantify MNAs in the smartphone signals: signal slope changes, turning point ratio (TPR) changes, and kurtosis changes. To detect AF beats among MNA-free segments, we use Root Mean Square of Successive RR Differences (RMSSD) and Shannon Entropy (ShE) which have been shown to be accurate in separating between NSR and AF patterns [1, 2, 16–21]. We set a decision boundary between MNA-corrupted and MNA-free signals, or between NSR and AF, using a support vector machine (SVM) concept. The SVM, which is widely used in various fields due to its low computational complexity, is trained, and then tests unknown input segments [22]. We evaluated the performance of our proposed algorithm using the smartphone camera signals obtained from subjects having MNA-free NSR, MNA-free AF, MNA- corrupted NSR, and MNA-corrupted AF. The main contributions of this paper are summarized as follows:

• Finding of Distinctive Features between MNA-free and MNA-corrupted smartphone-based pulsatile time series: We analyze MNA-free and MNA-corrupted pulsatile time series obtained from smartphones to characterize their features. Here, both MNA-free and MNA-corrupted signals include NSR and AF signals. We consider i) slope changes, ii) Kurtosis changes, and iii) TPR changes in a pulsatile times series. A paired t -test is performed to evaluate differences in these features between MNA-free and MNA-corrupted signals.

• Classification of types of MNA: We further specify the types of MNA, e.g. either finger misplacement or hand movement. The algorithm classifies finger misplacement from hand movement and notifies users of specific corrective actions, e.g. correct finger placement or minimal movement of the smartphone during measurements.

• Adjustable AF Detection Algorithm Implementation: Among identified MNA-free segments, AF beats are detected based on the RMSSD and ShE of pulse intervals. The algorithm operates in an adjustable and adaptive way so that it can be applied to the remained segments that are the subset of a pulsatile time series from which MNA segments are removed. It has a decision function which provides the start and end of MNA data segments to be removed. Moreover, the algorithm automatically extends the recording time when the length of the remaining segments is less than 2 minutes.

Fig. 1.

A smartphone application for acquiring heart rhythm information (the application gets heart rhythm using the camera lens and flash light).

II. Materials

A. Experimental Protocol

Smartphone signals were obtained from a fingertip using the video camera of an iPhone 4S, 5S or 6S. Smartphone data were collected respectively from MNA-free NSR, MNA-free AF, MNA-corrupted NSR, and MNA-corrupted AF segments of data taken from subjects recruited from the patients enrolled at the University of Massachusetts Medical Center (UMMC), the student communities of Worcester Polytechnic Institute (WPI), and the University of Connecticut (UConn). Specifically, MNA-free and MNA-corrupted AF recordings were obtained from 88 patients before cardioversion while MNA-free or MNA-corrupted NSR recordings were obtained from 88 patients after cardioversion as well as 11 healthy subjects. The 88 UMMC subjects who were enrolled were all confirmed to have AF pre- and non-AF or normal sinus rhythm post-cardioversion procedure. We have 88 MNA-free NSR, 88 MNA-free AF, and 11 MNA-corrupted recordings. The subjects all gave their informed consent (H-14490) (IRB numbers for the studies). During smartphone recording, all the subjects were asked to sit quietly and place their index or middle finger of their left or right hand on the smartphone’s camera lens.

For the evaluation of MNA discrimination performance, we tested our algorithm on MNA-corrupted data obtained from the 11 healthy subjects. We considered two types of MNA: hand movements and fingertip misplacement. These subjects did not have any known cardiovascular diseases. To induce the MNAs, healthy subjects were instructed to generate MNAs for 30 seconds within a 2-minute segment. For example, hand movement MNAs were controlled to be induced during 45–75 second period by moving a hand in up/down or left/right directions while holding smartphones. MNA amplitude varied for each subject as shown in Fig. 2. Finger misplacement MNA data, on the other hand, was collected by having the subject move their finger away from the smartphone’s lens 1–5 times within a 2-minute segment. Note the significant differences, e.g., slopes, amplitudes, and shapes, between corrupted and clean segments, or between hand movement MNA and fingertip misplacement MNA segments, as shown in Fig. 2.

Fig. 2.

Four raw sequences extracted from 60 beat-to-beat segments recorded by the smartphone camera. (a): MNA-free NSR, (b): MNA-free AF, (c) MNA-corrupted NSR (hand movement), (d) MNA-corrupted NSR (finger misplacement).

To evaluate AF detection performance, we tested our algorithm on AF data collected from 88 subjects having AF pre-cardioversion but who reverted to NSR after electrical cardioversion. AF rhythms have different features compared to NSR, as expected. Note also marked differences between AF and hand movement MNA data or between AF and fingertip misplacement MNA data, as shown in Fig. 2.

B. Preprocessing

Using iPhone 4S, 5S and 6S models, videos of fingertip blood flow intensity values were taken with 640×480 pixel resolution at a sampling rate of 30 frames per second and processed after 2 minutes of recording. Only the green band from the RGB video was used, as our recent results indicate this gives the best signal fidelity [23]. The intensity values of the upper 320×480 pixels in each frame were averaged, as our systematic analysis showed this region provided the best signal quality [1, 2]. Our approach first involves selection of only the MNA-free data from the 2-minute signal from which the AF detection is made. From these MNA-free segments, pulse beat-to-beat detection and pulse slope detection was performed after preprocessing including interpolation, sudden DC change elimination and two stages of a bandpass filter [2, 24]. These preprocessing steps are implemented by MATLAB.

III. Methods

Our algorithm includes AF and MNA discrimination capability that will further improve AF detection accuracy. The proposed AF detection algorithm for smartphone signals is fully implemented in the smartphone and is detailed in our flowchart shown in Fig. 3. It first discriminates and discards segments with fingertip misplacement MNA, then does the same for segments with hand movement MNA, and finally detects any AF among the MNA-free signals.

Fig. 3.

Flowchart of MNA discrimination and AF detection procedures

A. Feature Extraction

Features discriminating between MNA-corrupted and clean data are described in Section III-B while procedures for discriminating between AF and NSR are presented in Section III-C.

B. MNA Detection

1) Fingertip Misplacement MNA Detection

A frame from a video recording during NSR or an AF episode containing an entire fingertip image is shown in Fig. 4(a) and its successive corresponding time series is shown in Fig. 4(b). However, a frame from a video recording during a fingertip misplacement MNA episode containing a background image and without a fingertip image is shown in Fig. 4(c) and its successive corresponding time series is shown in Fig. 4(d). Based on this observation, we utilize the following metrics to detect finger misplacement MNA in smartphone recordings.

Fig. 4.

Different types of fingertip placement on a canera lens (left) with corresponding smartphone signals (right). (a)–(b) fingertip fully covers a lens: correct placement, (c)–(d) patially covered: misplacement

a) Color distribution changes

An iPhone provides YUV images recorded by a video camera. The YUV is a color space model consisting of one luma (or brightness), Y, and two chrominance components, U and V. We use the distribution of the Y of successive images to detect MNAs in smartphone signals. The gradient ΔS_{i, j} at the pixel (i, j) is derived as:

$$\mathrm{\Delta }Y=<Y_{i+1,j}-Y_{i,j},Y_{i,j+1}-Y_{i,j}>$$ (1)

where Y_{i, j} denotes the Y-luma at the pixel (i, j). The ΔY of a finger misplacement MNA is expected to be large at the pixels which are located at the boundary between background and fingertip images.

On the other hand, ΔY of NSR, AF, and hand movement MNA episodes are expected to be smaller at all the pixels.

b) Pulse amplitude changes

The pulse amplitude A_n,i at the i^th pulse of the n^th segment is defined by the difference between the i^th peak and the next (i + 1)^th trough amplitude values at the n^th segment. The smartphone signal of a background image without a fingertip image, caused by moving the fingertip entirely away from the lens (a type of finger misplacement MNA, in addition to only partial misplacement of the finger), is expected to have smaller amplitude when compared to that of NSR, AF and hand movement MNAs.

2) Hand movement MNA Detection

Our algorithm uses the three following parameters to detect hand movement:

• Slope ratio (maximum-minimum, standard deviation of maximum, and standard deviation of minimum)

• Turning point ratio (TPR)

• Kurtosis

NSR has a steeper slope in the systolic phase and a gradual slope in the diastolic phase. This is because a heart pumps blood out of the ventricles quickly but relaxes slowly. The heart of an AF subject pumps in an irregular way but maintains a higher speed of squeezing than that of relaxation. The steep slope in systole and the gradual slope in diastole have considerable similarities between NSR and AF episodes. Hand movement MNAs, in contrast, have different slope patterns as they may reflect blood intensity incurred not by spontaneous heart pumping but by hand movements. Moreover, finger misplacement MNAs have different patterns since they are mainly recording unwanted background images. The slope ΔC_n,i between the i^th sample and the next (i + 1)^th of the n^th segment can be calculated as follows:

$$\mathrm{\Delta }{\mathrm{C}}_{n,i}=C_{n,i}-C_{n,i-1}$$ (2)

where C_n,i is the peak-to-peak interval at the i^th sample of the nth segment. The positive maximal slope (P_n) and the negative minimal slope (N_n) in the n^th segment are given by

$$P_n=\mathrm{\Delta }{C}_{n,i}$$ (3)

$$N_n=\mathrm{\Delta }{C}_{n,i}$$ (4)

where S_n is a set of sample indices in the n^th segment. The max/min slope ratio R_n of the n^th segment is defined as:

$$R_n=|P_n/N_n|$$ (5)

where |x| denotes the absolute value of x. The R_n of NSRs and AFs is expected to be time-invariant while MNAs are time-varying. Since we inverted smartphone signals to facilitate peak detections, R_n of NSRs and AFs is smaller than a predefined threshold value (i.e. R_n ≪ 1) but the R_n values of MNAs are larger than the predefined threshold value.

The turning point ratio (TPR) is usually used to test the independency or count the number of magnitude changes in time series data. A turning point (TP) is defined as a local maximum or minimum point which is bigger or smaller than its two closest points. TPR is the number of TPs divided by the number of data points in a given segment. Hand movement MNA is expected to alter light intensity in video recordings at a higher frequency than the finger misplacement MNA does, due to the lens being rapidly covered and uncovered by the fingertip. This alteration in light intensity is expected to increase TPs or TPR in a time series. Hence, the TPR of hand movement MNA is expected to be larger than that of NSR and AF.

Kurtosis is used to measure the peakedness of a random variable and the kurtosis κ of the random variable X is defined as

$$k=E\left[{\left(\frac{X-\mu }{\sigma }\right)}^4\right]$$ (6)

where E [·] is expectation operator, μ denotes the expected value of X, and σ denotes standard deviation of X. The kurtosis of NSR and AF is expected to be small due to relatively even distribution coming from smooth and gradual changes in light intensity. On the other hand, the kurtosis of MNAs, which is calculated from the peaked and steep time series data, is expected to be larger than that of NSR and AF.

To determine a decision boundary between hand movement MNA-free and hand movement MNA-corrupted signals, a support vector machine (SVM) method is adopted [22].

C. AF Detection from MNA-Free Segments

We adopt our previous AF detection algorithm [16] which uses the statistics of pulse-to-pulse intervals (PPIs). Since a combination of the RMSSD and ShE of the PPIs yielded AF detection accuracy of 95 % in MNA-free smartphone signals [16], we adopt the same algorithms under the expectation that they would also provide similarly high AF detection accuracy in our MNA-eliminated smartphone signal segments.

Specifically, the RMSSD and ShE of the PPI time series a are calculated as described in Eqs. (7) and (8), respectively.

$$\mathit{\text{RMSSD}}(a_i,\dots ,a_{i+L-1})=\sqrt{\frac{1}{L}\sum _{j=0}^{L-1}{\{a_{i+j}-a_{i+j-1}\}}^2}.$$ (7)

where L denotes the length of a PPI segment.

$$\mathit{\text{ShE}}(a_i,\dots ,a_{i+L-1})=-\sum _{k=1}^{N_{\mathit{\text{BIN}}}}\frac{p(a_i,\dots ,a_{i+L-1},k)\text{ log }p(a_i,\dots ,a_{i+L-1},k)}{\text{log }(1/N_{\mathit{\text{BIN}}})},$$ (8)

where N_BIN denotes the number of bins which are equally spaced for the range of the PPI time series values in a disjointed way. In our experiment, we set N_BIN to 16. Each bin has upper (B_UP,k) and lower (B_LOW,k) boundaries where k ∈ [1, N_BIN]. p(a_i, …, a_i+L−1, k) is given in Eq. (9).

Our previous algorithm [16] compares the RMSSD and ShE values of 60 beat-to-beat PPI segments to their corresponding thresholds, respectively. As opposed to using a fixed length of 60 beat-to-beat segments, however, our new algorithm takes various ranges of PPI segment lengths as an input unless the length is less than a certain threshold, e.g., 20 beat-to-beats segments. If one of either the RMSSD and ShE values exceeds their thresholds, the pulsatile time series is classified as AF (see the third condition in the flowchart of Fig. 3).

IV. Results

We evaluated the performance of the MNA-tolerant arrhythmia discrimination algorithm with iPhone data. The baseline recording duration was two minutes. Our algorithm post-processed the recorded signal and eliminated MNA-corrupted data points. The recording duration was extended if the number of MNA-free beats after MNA elimination was below a predefined threshold number of 20 beats. We compared our algorithm to the previous AF algorithm which does not have MNA removal capability. As performance metrics, the accuracy, sensitivity, and specificity are considered.

A. Detection of MNA

1) Fingertip misplacement MNA

Our algorithm first checks whether a subject’s fingertip is correctly placed on the camera lens or not. Fig. 4 shows the pulse amplitude variations derived from smartphone signals.

Note especially that the image having only a uniform background without a fingertip (see Fig. 4c) gives relatively more irregular pulse amplitudes when compared to smartphone signals obtained with correct fingertip placement (see Fig. 4a). Our proposed algorithm prescreens these finger misplacement MNAs by using 2-dimensional image data. This is much easier than using pulsatile time series signals since there is so much fluctuation in the finger-misplaced MNA pulsatile time series signals, which can be incorrectly classified as pulse peaks.

We evaluated our algorithm’s performance on finger misplacement MNA detection by applying it to smartphone signals containing MNA-free NSR, MNA-free AF, hand movement MNA, and fingertip misplacement MNA. Fig. 5 shows the color distributions of these signals among which only fingertip misplacement MNA signal gives recognizable difference. We further investigated three different types of fingertip misplacement MNAs in Fig. 6c–h in terms of distribution of ΔY: 1) the background is recorded more than the fingertip, 2) the background is recorded less than the fingertip, and 3) only the background without a fingertip at all is recorded in the image. Using the mean, summation, and distribution of ΔY, the detection accuracy of the fingertip misplacement MNA was found to be 100% by our proposed algorithm.

Fig. 5.

Representative smartphone camera images (left) with their corresponding luma distributions (Y). (a)–(b) MNA-free NSR, (c)–(d) hand movement MNA, and (e)–(f) finger misplacement MNA.

Fig. 6.

Examples of representative fingertip placements on a smartphone lens (left) with their corresponding distributions of luma differences (ΔY) (right). (a)–(b) lens fully covered by a fingertip: correct placement, (c)–(d) more than half covered: misplacement, (e)–(f) less than half covered: misplacement, and (g)–(h) uncovered: misplacement

2) Hand movement MNA

Among the acquired smartphone data, after removal of fingertip misplacement MNA, we were left with 122 clean NSR segments, 100 clean AF segments, and 15 hand movement MNA-corrupted NSRs, each of which segments had a data length of 14 sec. The clean NSRs and clean AF have normal and regular pulse shapes during systole and diastole (see Figs. 2a and 2b) while the hand movement MNA-corrupted NSRs (see Fig. 7) contain abnormal and irregular pulse shapes as shown in Fig. 2c.

Fig. 7.

Hand movement in recording pulsatile signal from a finger using smartphone’s camera.

Fig. 8 compares maximum/minimum slope ratio, TPR, kurtosis, standard deviations of max and min slopes of NSR, AF, and hand movement MNA. Figs. 8a, 8c, 8e are two-dimensional plots while Figs. 8b, 8d, 8f are three-dimensional plots of Figs. 8a, 8c, 8e, respectively. We adopted the SVM method to detect hand movement MNAs emanating from smartphone signals [22]. In Fig. 8, the training data consists of 1,200 sets of the parameters (maximum/minimum slope ratio, TPR, kurtosis, standard deviations of max and min slopes) from each of the NSR, AF, and MNA categories. On the other hand, the remaining sets are used for validation. However, the implementation of the proposed algorithm actually consists of two stages as shown in Fig. 3: 1) MNA detection and 2) AF detection.

Fig. 8.

Parameter value comparison between NSR, AF, and hand movement MNA signals. (a) Maximum-minimum ratio (x axis) and turning point ratio (y axis), (b) three dimension plot of (a), (c) standard deviation of maximum (x axis) and minimum (y axis) slope value, (d) three dimension plot of (c) in logarithmic scale, (e) kurtosis (x axis) and standard deviation of minimum slope value (y axis), and (f) three dimension plot of (e).

Hence, for MNA detection, we used the same number of training sets from MNA and non-MNA training (NSR or AF). As discussed in Section II, max/min slope ratio, TPR, kurtosis, and standard deviation of maximum (or minimum) slope values are significantly different between hand movement MNA and the other categories (NSRs and AFs). Hence, the SVM boundaries between hand movement MNAs and the other categories using these statistical metrics are recognizable, as shown in Figs. 8a–d. However, kurtosis between MNA and the other categories are not significantly different, as shown in Figs. 8e–f. However, all three statistical values are similar for AF and NSR. Using SVM training, we derived the optimal boundary for distinguishing between hand movement MNA and the other types of smartphone signals via the segment lengths L_{slope_max} = 1s, L_{slope_min} = 1s, L_tpr = 1s, L_kur = 1s, L_{std_dev_slope_max} = 14s, and L_{std_dev_slope_min} = 14 s.

3) MNA Detection Performance among NSR signals with and without MNA

To verify our algorithm, we collected iPhone recordings from 7 healthy subjects without any history

$$p(a_i,\dots ,a_{i+L-1},k)=\sum _{j=0}^{L-1}U(a_{i+j},k)/(L-N_{\mathit{\text{outliers}}})\text{ for }U(a_{i+j},k)=\{\begin{array}{cc}1,\hfill & B_{\text{LOW},k}<a_{i+j}<B_{\text{UP},k}\hfill \\ 0,\hfill & \text{otherwise}\hfill \end{array}$$ (9)

of cardiovascular disease. Each subject was asked not to move their measurement hand for the first 30 seconds. For the next 30 seconds, each subject was asked to move their hand. For the last 30 seconds, each subject was asked to misplace their fingertip.

Each data type was sequentially categorized into “MNA-free”, “MNA Type 1-corrupted”, and “MNA Type 2-corrupted.” MNA Type 2-corrupted NSR (hand misplacement) is correctly detected with an accuracy of 100%. Considering only MNA-free NSR and MNA Type 1-corrupted NSR, we found the accuracy, sensitivity, and specificity to be 0.9698, 0.9812, and 0.9584.

Table I shows true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) of AF detection before and after MNA elimination on seven non-AF subjects. In subject II, for example, an incorrect AF detection was reported 110 times out of 819 segments before MNA removal, but with our MNA detection algorithm, it was determined that 69% of the segments were contaminated by MNA. Thus, when these segments were analyzed by our AF detection algorithm, there was a smaller number of false positives, e.g. 23 false positives, by removing MNA-corrupted segments beforehand. Similarly, for subjects I and III–VII, all false positives AF detection rates were reduced to zero because they were due to MNA and not AF. To reiterate, the concept is that if MNAs are detected in a data segment, then the algorithm will ignore the segment, thereby reducing false positives (FPs). In summary, inclusion of our MNA algorithm increased the specificity from 73% to 97%.

TABLE I.

True Positives, True Negatives, False Positives, and False Negatives Before and After MNA Removal Based on NON-AF Subject Data Sets

Subject	with original segments				after MNA removal				Elimination rate (MNA detection rate)
	True positives	True negatives	False positives	False negatives	True positives	True negatives	False positives	False negatives
I	0	24	51	0	0	24	0	0	68.00 %
II	0	709	110	0	0	653	23	0	17.46 %
III	0	22	50	0	0	21	0	0	70.83 %
IV	0	12	33	0	0	12	0	0	73.33 %
V	0	20	34	0	0	19	0	0	64.81 %
VI	0	5	4	0	0	5	0	0	44.44 %
VII	0	6	12	0	0	6	0	0	66.67 %
Overall	0	1092	294	0	0	740	23	0
	specificity: 0.7308 sensitivity: N/A				specificity: 0.9699 sensitivity: N/A

At the bottom of Table I, the overall sensitivity value was calculated to be N/A. The sensitivity is calculated by TP/(TP + FN) where TP and FN in the AF detection indicate AF subjects correctly identified as AF, and AF subjects incorrectly identified as NSR, respectively. Since we consider only NSR subjects in Table I, the number of AF subjects is zero, which results in the sensitivity value to be N/A.

B. Detection of AF from MNA-Free SmartPhone Data

Table II is a confusion matrix of our proposed algorithms on AF, NSR, MNA Type 1 and MNA Type 2 subjects. For the 91 NSR subjects, 6 false negatives (FNs) were changed to 1 FN after the use of our algorithm; thus, specificity increased from 93.41% to 98.84%. Table III compares the AF detection performance of the conventional and proposed algorithms.

TABLE II.

Confusion Matrix of Our MNA-Resilient AF Detection On AF, NSR, MNA type 1, MNA type 2 Subjects

		Predicted Class
		AF	NSR	MNA type 1	MNA type 2
Actual Class	AF	87	3	1	0
	NSR	2	83	6	0
	MNA type 1	0	0	7	0
	MNA type 2	0	0	0	7

TABLE III.

Comparison of AF Detection Performance between Conventional and Proposed Algorithms

	AF Detection without MNA Discrimination [11]	AF Detection with MNA Discrimination
Sensitivity	0.9560	0.9667
Specificity	0.7905	0.9765
Accuracy	0.8673	0.9714

Using our proposed algorithm with SVM, the accuracy, sensitivity, and specificity increased from 0.9560, 0.7905, and 0.8673 to 0.9667, 0.9765, and 0.9714, respectively. For AF detection, we used the same number of training sets from NSR and AF.

C. Contactless Measurement of Heart Rhythm

We applied contactless method in detecting heart rhythm and evaluate the method by comparing it to the contact-based (fingertip-based) method described in Sections III–IV. Specifically, subjects are asked to show a palm side of their hand to a camera which is located on a desk while holding a smartphone while placing their index or middle finger on a camera lens as shown in Fig. 9. For the contactless method especially, the measurement was performed in the laboratory with its lights turned on and the distance between the palm and the camera is controlled to be 25cm.

Fig. 9.

Contactless and contact-based (fingertip-based) methods for heart rhythm recordings.

The sampling rate of camera recording is set to 30 fps for both contactless and contact-based methods. Since the contactless and contact-based methods are applied at the same time, measured heart rhythm could be compared in a convenient way.

From the simultaneous 10 contactless and contact-based measurements, the mean and standard deviation values of the heart rate difference between two methods are 0.46 and 1.42 and the correlation coefficient value is 0.98.

V. Conclusion

Finger misplacement and hand movement while using a smartphone application are inevitable, especially for elderly subjects as they may not be accustomed to using smartphones or may have uncontrollable tremors. Our algorithm was designed to detect if a subject’s finger is either misplaced or is subject to significant motion. To distinguish these features, we make use of image distributions and light intensity from the successive camera images. If the intensity difference between neighboring pixels is larger than the pre-defined threshold, then our algorithm rejects the signal and notifies the user to place his or her finger back over the smartphone’s camera. On the other hand, if the pulse interval signal features such as max/min slope changes, TPR, or kurtosis obtained from the average of light intensity are out of the threshold ranges, then the algorithm decides that the segment is corrupted by motion and noise artifacts. As a result of these strategies, we have enhanced the AF detection performance compared to our earlier method, by reducing these types of MNAs before AF detection. Moreover, the limitation of the previous algorithm was that it cannot be applied to a variety of lengths of smartphone recordings because a decision boundary distinguishing NSR from AF was derived only from 120-second length signals. In this paper, we overcome this limitation of the previous algorithm by adapting its AF decision boundary to different lengths of recordings.

We have also studied on a contactless heart rate detection method which detects heart rate from a hand using a camera. We have evaluated the performance of the contactless method by comparing it to the contact-based (or fingertip-based) method. As a result, the heart rate difference between contactless and contact-based methods was observed to be 0.46±1.42, which shows the potentiality of the contactless method being used for the MNA-resilient AF detection, too.

Given the growing popularity of smartphones and their potential for medical diagnostic applications, our approach to AF detection using a smartphone has definite appeal to both physicians and patients. A smartphone application with embedded MNA detection algorithms as described in this paper will ultimately result in better sensitivity and specificity which will lead to return to better adoption of this technology for AF monitoring. A clinical trial testing the proposed technology for AF detection with more subjects is currently ongoing.