High-Throughput, Contact-Free Detection of Atrial Fibrillation From Video With Deep Learning

Abstract

This study uses video and a pretrained deep convolutional neural network to analyze facial photoplethysmographic signals in detection of atrial fibrillation.

Approaches for atrial fibrillation (AF) detection can screen only 1 patient at a time.¹ In 2018,² we demonstrated a novel method of AF detection by analyzing facial photoplethysmographic (FPPG) signals without physical contact using a smartphone camera.² In this proof-of-concept study, we prospectively evaluated the feasibility of high-throughput AF detection by analyzing FPPG signals³ from multiple patients concurrently using a single digital camera and a pretrained deep convolutional neural network (DCNN).⁴

Methods

After institutional approval from the Joint Chinese University of Hong Kong–New Territories East Cluster Clinical Research Ethics Committee and individual written informed consent, 20 patients (mean [SD] age, 76.6 [7.6] years; 12 men [60%]) with permanent AF and 24 control individuals (mean [SD] age, 56.8 [20.2] years; 14 men [58.3%]) in sinus rhythm (SR) were recruited. A digital camera (50D; Canon) was used to film 5 patients sitting in a row 150 cm away (Figure). We recorded 64 videos (1-minute duration, 24 FPS), each capturing 5 patients simultaneously in 32 different heart-rhythm permutations based on a 5-participant binary (AF/SR) matrix and repeated once using patients chosen at random. Patients were instructed to keep their head stationary and not talk. The FPPG signals from patients were automatically extracted from the videos, resampled to 30 Hz, and analyzed in segments of 512 samples using Cardiio Deep Rhythm (Cardiio Inc), a DCNN previously trained for detecting AF from PPG waveforms.⁴ Pulse irregularity in more than 50% FPPG segments for each patient was considered positive for AF. The investigator applying DCNN was blinded to the reference electrocardiogram (ECG) and participant binary matrix.

Figure. Experimental Setup.

A, Example video recording of 5 patients arranged in 1 of the 32 different heart rhythm permutations with extracted facial photoplethysmographic (FPPG) signals and reference electrocardiogram (ECG) traces. B, The 5-participant binary (atrial fibrillation [AF]/sinus rhythm [SR]) matrix.

We calculated Cohen κ coefficients to assess the agreement and conducted a generalized linear mixed model with AF/SR as dependent variable, using a logit link, to control for seating position and heart-rhythm permutation as fixed effects and patients as random effect. All P values less than .05 were considered significant, and all P values were 2-sided.

Results

On average, each patient appeared in 7 videos. Of the 44 patients, the test-retest reliability of FPPG was 95.4% with κ = 0.91 (95% CI, 0.78-1.00). The generalized linear mixed model demonstrated very good reliability and consistent results between patients (intraclass correlation coefficient = 0.88; P = .004), irrespective of seating position and heart-rhythm permutation. A total of 320 individual FPPG signals were analyzed (Table). Agreement between FPPG and ECG was excellent (95.9%), with κ = 0.92 (95% CI, 0.88-0.96) varying between 0.84 and 1.00 across seating positions. Overall sensitivity was 93.8% (95% CI, 88.9%-96.6%); specificity, 98.1% (95% CI, 94.6%-99.4%); positive predictive value, 98.0% (95% CI, 94.2%-99.4%); and negative predictive value, 94.0% (95% CI, 89.6%-96.6%) in discriminating AF from SR. Heart rhythms of all 5 patients were correctly identified in 79.7% of videos (n = 51 of 64) while only 1 patient was misclassified in the remaining 20.3% (n = 13 of 64). Possible causes for false-positive results (n = 3) included motion artifact and ectopic beats, whereas slow AF with ventricular rates less than 60 bpm was the main possible cause for false-negative results (n = 9 of 10). Receiver operating characteristic analysis yielded a C statistic of 0.99; thresholds of 0.33, 0.91, and 0.67 were optimized for sensitivity (sensitivity 100% and specificity 85%), specificity (sensitivity 84.4% and specificity 99.4%), and both sensitivity and specificity (sensitivity 95.6% and specificity 96.2%), respectively.

Table. Diagnostic Accuracy of Facial Photoplethysmographic in Detecting Atrial Fibrillation.

Variable	12-Lead ECG		Total
	AF Present	AF Absent
FPPG, No. (%)a
Positive	150 (46.9)	3 (0.9)	153
Negative	10 (3.1)	157 (49.1)	167
Total No.	160	160	320

Abbreviations: AF, atrial fibrillation; ECG, electrocardiogram; FPPG, facial photoplethysmographic.

^aThe percentages were the number in each cell divided by the total number of tests.

Discussion

To our knowledge, this is the first study to demonstrate detection of AF with high accuracy from multiple patients concurrently with a single camera. Our findings raise the possibility that this low-cost approach may be scalable for high-throughput AF screening. The proposed approach circumvents the need to perform serial 1-on-1 screening and requires minimal effort from patients, potentially saving time and reducing work for clinical staff. Limitations include the need for patients to stay still for a minute and confirmatory ECG for suspected AF. In practice, the positive predictive value will inevitably be lower because AF prevalence in the general population is lower than in this proof-of-concept study. Importantly, this technology must be deployed in a manner that respects patient privacy (eg, having a clearly designated seating area in the clinic for opt-in screening). Further studies are warranted to evaluate performance in a real-world clinical and home setting.