Identifying Atrial Fibrillation With Sinus Rhythm Electrocardiogram in Embolic Stroke of Undetermined Source: A Validation Study With Insertable Cardiac Monitors

Author's summary

Paroxysmal atrial fibrillation (AF) is a potential cause of embolic stroke of undetermined source (ESUS). Because oral anticoagulation therapy can reduce the risk of recurrent stroke, identifying hidden AF is essential. This is the first study that validated a deep learning algorithm (DLA) identifying paroxysmal AF using implantable cardiac monitors, which is accepted as a gold standard for diagnosing paroxysmal AF in patients with ESUS. Furthermore, we provide a useful AF prediction model by combining clinical factors, including burden of atrial ectopy, left atrial diameter and the DLA. This model might be clinically useful in management of ESUS patients.

Abstract

Background and Objectives

Paroxysmal atrial fibrillation (AF) is a major potential cause of embolic stroke of undetermined source (ESUS). However, identifying AF remains challenging because it occurs sporadically. Deep learning could be used to identify hidden AF based on the sinus rhythm (SR) electrocardiogram (ECG). We combined known AF risk factors and developed a deep learning algorithm (DLA) for predicting AF to optimize diagnostic performance in ESUS patients.

Methods

A DLA was developed to identify AF using SR 12-lead ECG with the database consisting of AF patients and non-AF patients. The accuracy of the DLA was validated in 221 ESUS patients who underwent insertable cardiac monitor (ICM) insertion to identify AF.

Results

A total of 44,085 ECGs from 12,666 patient were used for developing the DLA. The internal validation of the DLA revealed 0.862 (95% confidence interval, 0.850–0.873) area under the curve (AUC) in the receiver operating curve analysis. In external validation data from 221 ESUS patients, the diagnostic accuracy of DLA and AUC were 0.811 and 0.827, respectively, and DLA outperformed conventional predictive models, including CHARGE-AF, C2HEST, and HATCH. The combined model, comprising atrial ectopic burden, left atrial diameter and the DLA, showed excellent performance in AF prediction with AUC of 0.906.

Conclusions

The DLA accurately identified paroxysmal AF using 12-lead SR ECG in patients with ESUS and outperformed the conventional models. The DLA model along with the traditional AF risk factors could be a useful tool to identify paroxysmal AF in ESUS patients.

Graphical Abstract

INTRODUCTION

Paroxysmal atrial fibrillation (AF) is a potential cause of embolic stroke of undetermined source (ESUS).¹⁾ Oral anticoagulation therapy can considerably reduce the risk of recurrent AF-related strokes. Therefore, identifying hidden AF is essential for secondary prevention.²⁾ Although insertable cardiac monitors (ICMs) achieve approximately 100% AF detection rate,³⁾ they are expensive and invasive, which limits their adoption in clinical practice. Several scoring systems with relevant clinical risk factors have been proposed but these models exhibit moderate performance.⁴⁾,⁵⁾,⁶⁾ Thus, there are unmet needs to precisely identify hidden AF with low cost and non-invasive tests.

Deep-learning-based electrocardiogram (ECG) interpretation allows the prediction of some diseases or conditions, such as left ventricular dysfunction, valvular heart disease, anemia, and presence of electrolyte abnormalities or renal impairment.⁷⁾,⁸⁾,⁹⁾ Several deep learning algorithms (DLAs) have been developed to identify paroxysmal AF with ECG during sinus rhythm (SR). Although ECG DLA models outperformed other traditional risk factors and conventional prediction models, the accuracy of such models is not sufficient to use in clinical practice.¹⁰⁾ Combining the model with other clinical risk factors may enhance AF prediction accuracy. In this study, we aimed to develop new DLA for predicting paroxysmal AF with 12-lead ECG during sinus rhythm and validated the model for identifying hidden AF in ESUS patients who underwent ICM insertion. We also sought to optimize the DLA model combined with the clinical risk factors and devise the explainable modality for DLA.

METHODS

Ethical statement

This study was approved by the Institutional Review Boards (IRBs) of Sejong General Hospital (Bucheon, Korea; IRB No. 2019-0411), Mediplex Sejong Hospital (Incheon, Korea; IRB No. 2019-083), and Seoul National University Bundang Hospital (Seongnam, Korea; IRB No. B-2204-753-105).

Deep learning algorithm model development

Study population and design

In this study, patients aged ≥18 years in Sejong General Hospital and Mediplex Sejong Hospital were studied. Retrospective data was collected from patients with AF and those without AF who had undergone ECG during the study period of 2016–2021. The ECG results were reviewed by cardiologists, and trained registered nurses labeled each ECG for the presence or absence of the arrhythmia. The development data consisted of sinus rhythm ECGs from AF patients with at least one instance of AF rhythm and non-AF patients who had no reference to AF in their ECGs or electronic medical records during the study period. Digitally stored 12-lead ECG data, age, and sex were considered as predictors for AF. ECG data were recorded for 10 seconds at 500 points per second (500 Hz). Because more artifacts existed than other areas, 1 second each was removed at the beginning and end of the ECG. A dataset using the remaining 12-lead ECG data was established and randomly split in a 9:1 ratio for model developing and internal validation. The training and test datasets are separated by patient, so there is no duplication where a patient’s ECG is used in both datasets. In the external validation, the final model was tested in ESUS patients with ICM. The diagnostic performance was compared with conventional AF prediction model and known risk factors. A flowchart of the study process is displayed in Figure 1.

Figure 1. Study flow.

AF = atrial fibrillation; DLA = deep learning algorithm; ECG = electrocardiogram; ESUS = embolic stroke of undetermined source; ICM = implantable cardiac monitoring.

Model development

The architecture of proposed DLA model consists of three modules. Module 1 is responsible for extracting ECG-derived information, while module 2 extracts demographic data. The third module then integrates the two sets of information to predict AF. Figure 2 describes the model structure which is motivated by ResNet and utilized in previous study.¹¹⁾ The latent features of ECGs are represented in module 1, which consists of several residual blocks, each of which has two submodules. Each submodule consists of two sets of one-dimensional convolutional neural network (Conv1D), batch normalization (BatchNorm1D), and rectified linear unit activation (ReLU), and a dropout layer on the back. The first submodule has one skip connection with Conv1D and Max Pooling layers. The second block has only skip connection. Module 2 has a single fully linked layer (FC) that extracts additional latent features for patient information such as age, gender, weight, and height. Missing values were imputed with zero. Module 3 also contains a single FC for determining AF probability (DLA probability) by combining two latent features.

Figure 2. Architecture of the deep learning algorithm model.

AF = atrial fibrillation; ECG = electrocardiogram; BatchNorm1D = one-dimensional batch normalization; Conv1D = one-dimensional convolutional neural network; ReLU = rectified linear unit activation.

External validation with embolic stroke of undetermined source patients

Validation cohort

The Seoul National University Bundang Hospital ICM registry data was used an external validation cohort. This registry includes all-consecutive ESUS patients who underwent ICM insertion for AF surveillance from March 1, 2016, to July 31, 2021. The participants were followed up from the ICM procedure date to April 15, 2022, ICM explantation, or death whichever came first.

The ESUS was diagnosed according to the global consensus by neurologists.¹²⁾ The diagnostic criteria of ESUS includes 1) non-lacunar infarction, 2) patent large arteries (>50%) supplying the area of ischemia, 3) no major risk of cardioembolic source (AF, severe left ventricular dysfunction), 4) absence of other specific causes of stroke (e.g., arteritis, dissection, migraine/vasospasm, drug misuse). Patients with no available follow up ICM data, prior history of AF/flutter, or other specific causes of intracardiac thrombosis (e.g., pulmonary vein thrombosis after pneumonectomy) were excluded.

All commercially available ICM devices, including Reveal XT/LINQ™ (Medtronic, Minneapolis, MN, USA), Confirm RX™ (Abbott, Minneapolis, MN, USA), and Biomonitor III (BIOTRONIK SE & CO. KG, Berlin, Germany), were used for AF screening. All stored AF events ECGs in ICM were independently reviewed by 2 electrophysiologists.

The baseline 12-lead ECGs first recorded during the hospitalization for acute stroke were applied to the DLA. The patients who had AF episodes ≥5 minutes by ICM were labeled as the AF group. Here, 5 minutes is the shortest AF duration that could be related to the increase in the risk of stroke based on previous studies.²⁾

Measurements of left atrial diameter, left atrial volume index, and atrial ectopic burden

Left atrial diameter (LAD) and left atrial volume index (LAVI) was measured using echocardiography performed during the hospitalization for acute strokes. Here, LAD was defined as the length between anterior and posterior wall of LA at ventricular end-systole in parasternal long-axis view at ventricular end-systole. The LAVI was calculated by dividing the LA volume (mL) by body surface area (m²). According to a previous study, the atrial ectopic burden (AEB) was calculated by 100×number of conducted QRS complexes from atrial ectopy/total number of QRS complex during 24-hour Holter monitoring.¹³⁾

Conventional atrial fibrillation prediction models

Several AF risk-stratification models have been developed and validated. These models utilized obtainable clinical variables, such as age, ethnicity, height, weight, blood pressure, smoking status, hypertension, diabetes, thyroid disease, heart failure, and myocardial infarction. To compare the diagnostic accuracy of the DLA, the AF risk score of conventional prediction models, including CHARGE-AF,⁶⁾ C2HEST,¹⁴⁾ and HATCH¹⁵⁾ in external validation cohort, were calculated. The detailed variables and formulas are described in Supplementary Data 1.

ShapeExplainer for explainable artificial intelligence

ShapeExplainer is a method based on the generative adversarial network to explain the deep learning classification model, which is a black box.¹⁶⁾,¹⁷⁾ ShapeExplainer can convert SR ECG without paroxysmal AF to synthesized SR ECG with paroxysmal AF by using the trained DLA. By comparing the original ECG with the synthesized ECG, the region of interest and morphological shape of three distinct waves of ECG that DLA focuses on can be analyzed. ShapeExplainer provides qualitatively model interpretation by presenting the shape changes of synthesized ECG as the counter factor.

Statistical analysis

Continuous variables are presented as mean (standard deviation) and compared using the unpaired Student’s t-test or Mann–Whitney U test. Categorical variables are expressed as frequencies and percentages and compared using the χ² test. At each input of validation data, the DLA calculated the probability of paroxysmal AF in the range from 0 to 1. The performance of DLA and other model was evaluated by presenting general performance metrics; positive predictive value, negative predictive value, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) curve. If data did not conform to a normal distribution and were highly skewed (e.g., AEB), the values were log-transformed for logistic regression and receiver operating characteristic (ROC) analysis. Univariate and multivariate logistic regression analyses were performed to identify proportional hazard risk for paroxysmal AF and adjust for known potential confounders: DLA >0.5, log (AEB), left atrium (LA) diameter, PR interval, and CHA₂DS₂-VASc score.¹⁸⁾ Statistical significance was set at p value <0.05. All statistical analyses were performed using IBM/SPSS v24.0 (IBM Corp., Armonk, NY, USA) and RStudio (Integrated Development Environment for R; formerly RStudio, PBC, Boston, MA, USA). We used Pytorch’s open-source software library as the backend, and Python for the analysis.

RESULTS

Development of the deep learning algorithm for atrial fibrillation prediction

A total of 44,085 SR ECGs (12,932 ECG from AF patients and 31,153 ECG from non-AF patients) were included as the development data of DLA for AF prediction during SR from Hospital A and B (Figure 1, Supplementary Table 1). The median time difference between individual AF ECG and SR ECG was 59 days (interquartile range [IQR], 6–359). In the internal validation, the AUROC of the DLA detecting AF during SR was 0.862 (95% confidence interval [CI], 0.850–0.873) and the area under Precision-Recall curve (AUPRC) was 0.675 (95% CI, 0.651–0.701). The sensitivity, specificity, positive predictive value, and negative predictive value of the DLA were 0.814, 0.767, 0.575, and 0.914, respectively (Supplementary Table 2, Supplementary Figure 1).

Validation of the deep learning algorithm in embolic stroke of undetermined source patients

A total of 221 ESUS patients with ICM was included as an external validation cohort. The median duration of ICM was 14.3 months (IQR, 8.1–22.3 months) and the AF, sustained more than 5 minutes, was detected in 34 patients (15.4%). The patients with AF had older age (70.4 vs. 64.0, p=0.003), larger LA diameter (41.1 vs. 35.7 mm, p<0.001) and higher AEB (0.2% vs. 0.02%, p<0.001). The DLA probability was significantly higher in patients with AF (0.639 vs. 0.406, p<0.001). The detailed baseline characteristics are described in Table 1.

Table 1. Baseline characteristics of study population.

Characteristics		No AF detection (n=187)	AF detection (n=34)	p value
Age (years)		64.0±13.9	70.4±10.5	0.003
Female		64 (34.2)	10 (29.4)	0.156
Height (cm)		164.1±9.2	163.2±9.8	0.599
Weight (kg)		65.6±10.9	67.6±13.2	0.354
BMI (kg/m2)		24.3±3.2	25.2±3.1	0.129
Duration from stroke to ICM implantation (months)		1.7 (0.2, 7.7)	0.8 (0.2, 3.4)	0.041
Past history
	Heart failure	1 (0.5)	4 (11.8)	0.002
	Hypertension	102 (54.5)	23 (67.6)	0.156
	DM type 2	52 (27.8)	13 (38.2)	0.220
	HCMP	0 (0.0)	2 (5.9)	0.023
	Myocardial infarction	1 (0.5)	1 (2.9)	0.173
	COPD	5 (2.7)	0 (0.0)	0.335
	Hyperthyroidism	4 (2.1)	0 (0.0)	0.389
Smoking				0.774
	Never smoker	116 (62.0)	22 (64.7)
	Ex-smoker	34 (18.2)	7 (20.6)
	Current smoker	37 (19.8)	5 (14.7)
CHA2DS2-VASc		3.7±1.3	4.4±1.3	0.002
Discharge medication
	Aspirin	150 (80.2)	21 (61.8)	0.018
	Clopidogrel	91 (48.7)	15 (44.1)	0.710
	OAC	16 (8.6)	7 (20.6)	0.035
	Statin	170 (90.9)	27 (79.4)	0.047
ECG parameters
	Heart rate (bpm)	71.0±13.3	68.3±12.3	0.310
	PR interval on ECG (ms)	174.3±27.1	187.1±40.5	0.021
	QRS width (ms)	92.4±13.3	96.1±17.7	0.184
	QT interval (ms)	400.0±37.2	414.5±42.0	0.052
Holter monitoring parameters
	Number of APCs	19 (7, 69)	170 (34, 601)	<0.001
	Burden of atrial ectopy (%)	0.02 (0.01, 0.08)	0.20 (0.04, 0.73)	<0.001
Echocardiography parameters
	LVEF (%)	63.1±4.2	62.8±5.5	0.707
	LA diameter (mm)	35.7±5.0	41.1±5.0	<0.001
	LAVI (mL/m2)	32.2±8.9	46.1±17.2	<0.001
	PFO	53 (28.3)	10 (29.4)	0.899

Values are expressed as number (%), mean ± standard deviation or median (interquartile range).

APC = atrial premature contraction; BMI = body mass index; COPD = chronic obstructive pulmonary disease; DM = diabetes mellitus; ECG = electrocardiogram; HCMP = hypertrophic cardiomyopathy; ICM = implantable cardiac monitoring; LA = left atrium; LAVI = left atrial volume index; LVEF = left ventricular ejection fraction; OAC = oral anticoagulant; PFO = persistent foramen ovale.

The diagnostic accuracy of the DLA for identifying AF in ESUS patients was 0.811 (95% CI, 0.809–0.812), and AUROC was 0.827 (95% CI, 0.743–0.912) (Figure 3). The sensitivity, specificity, positive predictive value, and negative predictive values of the DLA were 0.824, 0.824, 0.438, and 0.962, respectively (Table 2, Supplementary Table 3). When training logistic regression with the DLA probability, AEB, and LAD, we achieved the best performance with an AUROC of 0.906 (95% CI, 0858–0.955). The diagnostic performance of DLA was superior to that of the conventional prediction models based on clinical factors, including CHARGE-AF, C2HEST, and HATCH (Figure 3, Table 2).

Figure 3. Receiver operating characteristic curves in external validation cohort.

AEB = atrial ectopic burden; AF = atrial fibrillation; AUC = area under the curve; DLA = deep learning algorithm; ESUS = embolic stroke of undetermined source; LAD = left atrium diameter.

Table 2. Predictive performance for detection of the atrial fibrillation according to models.

Prediction models	AUC	Sensitivity	Specificity	PPV	NPV	F1 score
DLA with Log AEB and LAD	0.906 (0.858–0.955)	0.862 (0.737–0.988)	0.836 (0.782–0.891)	0.463 (0.330–0.596)	0.974 (0.948–0.999)	0.602 (0.465–0.743)
DLA	0.827 (0.743–0.912)	0.824 (0.700–0.967)	0.807 (0.742–0.856)	0.438 (0.276–0.518)	0.962 (0.940–0.996)	0.572 (0.396–0.675)
Log (AEB)	0.807 (0.731–0.883)	0.724 (0.561–0.887)	0.785 (0.725–0.846)	0.356 (0.234–0.478)	0.946 (0.909–0.982)	0.477 (0.330–0.621)
LAD	0.775 (0.688–0.863)	0.793 (0.646–0.941)	0.633 (0.562–0.704)	0.261 (0.170–0.353)	0.949 (0.910–0.989)	0.393 (0.269–0.513)
CHARGE-AF	0.652 (0.548–0.755)	0.621 (0.444–0.797)	0.689 (0.621–0.757)	0.247 (0.148–0.345)	0.917 (0.870–0.964)	0.353 (0.222–0.482)
C2HEST	0.581 (0.481–0.680)	0.828 (0.690–0.965)	0.345 (0.275–0.415)	0.171 (0.109–0.234)	0.924 (0.860-0.988)	0.283 (0.188–0.377)
HATCH	0.570 (0.473–0.667)	0.793 (0.646–0.941)	0.367 (0.296–0.438)	0.170 (0.107–0.234)	0.915 (0.851–0.980)	0.280 (0.184–0.375)

AEB = atrial ectopic burden; AUC = area under the curve; CI = confidence interval; DLA = deep learning algorithm; LAD = left atrial diameter; NPV = negative predictive value; PPV = positive predictive value.

The AUROC were 0.807 (95% CI, 0.731–0.883) for AEB and 0.775 (95% CI, 0.688–0.863) for LAD (Table 2). According to ROC analysis, the best cutoff value for AF prediction was 38.5 mm with LAD and 0.050% with AEB. The 3-dimensional scatter plot revealed that AF detection clustered around the area, which was indicated by the higher DLA, AEB, and LAD (Figure 4). Multivariable analysis revealed that a DLA probability greater than 0.5, a cut-off value derived from our internal validation set using the Youden Index, emerged as the most crucial factor for predicting paroxysmal AF. This was followed by log (AEB) and LAD (Table 3).

Figure 4. Three-dimensional scatter plot of DLA and clinical features according to AF occurrence.

AF = atrial fibrillation; DLA = deep learning algorithm.

Table 3. Logistic regression analysis for detection of the atrial fibrillation.

Variables	Univariable		Multivariable
	OR (95% CI)	p value	OR (95% CI)	p value
DLA >0.5	10.80 (4.515–30.18)	<0.001	11.63 (3.979–41.51)	<0.001
Age (years)	1.040 (1.009–1.074)	0.013
Log (AEB)	3.933 (2.402–6.949)	<0.001	3.625 (1.928–7.371)	<0.001
LA diameter (mm)	1.236 (1.139–1.355)	<0.001	1.184 (1.066–1.332)	0.003
PR interval, (ms)	1.012 (1.001–1.024)	0.028	1.020 (1.004–1.038)	0.010
CHA2DS2-VASc	1.505 (1.143–2.008)	0.004	0.962 (0.641–1.430)	0.850

AEB = atrial ectopic burden; CI = confidence interval; DLA = deep learning algorithm; LA = left atrium; OR = odds ratio.

Explainable deep learning algorithm

The Figure 5 displays a representative case of ShapeExplainer. The blue-lined ECG is a SR ECG without underlying paroxysmal AF, and the orange-lined ECG is synthesized SR ECG with paroxysmal AF. The morphology of ECG can be changed from non-AF ECG to ECG with high DLA probability predicting paroxysmal AF. In this case, longer PR interval, higher QRS amplitude, and longer QT interval were noted in synthesized ECG, which has high probability of AF, compared with the original ECG.

Figure 5. Explainable deep learning algorithm providing synthesized paroxysmal AF ECG by ShapeExplainer.

AF = atrial fibrillation; ECG = electrocardiogram.

DISCUSSION

In this study, the DLA was developed to identify paroxysmal AF using 12-lead ECG during SR. The performance of the developed DLA was subsequently validated using ICM, which exhibited the highest efficacy for AF screening in patients with ESUS. A major strength of this study is that the best available cohort was used for external validation. The ICM allows continuous ambulatory ECG monitoring for 2–3 years. The sensitivities for AF detection of these devices were reported to be over 95%.¹⁹⁾ This study revealed that the diagnostic performance of the DLA for AF prediction was superior to conventional models. Furthermore, when combined with AEB and LAD, enhanced accuracy could be obtained for predicting paroxysmal AF in ESUS patients. To the best of our best knowledge, this study is the first to validate a DLA for predicting AF in high-risk patients with a golden standard AF screening device.

There is no wonder that DLA probability, LAD, and AEB are related to AF because these parameters are associated with the key components for AF development. AF triggers and atrial substrates play a major role in AF initiation and maintenance.¹⁸⁾ The baseline SR ECG can reflect the atrial abnormal substrates including chamber enlargement, fibrosis, and electrical remodeling.¹⁰⁾ Because the DLA was designed to calculate the similarity to SR ECG figures of patients with paroxysmal AF, the DLA could indirectly estimate the degree of atrial structural and electrical remodeling. The echocardiographic LA diameter is another reliable parameter that reflects the degree of atrial remodeling.¹⁸⁾ Finally, the AEB on Holter monitoring can reflect the burden of AF triggers.²⁰⁾ Thus, these parameters are complementary for AF identification. Our group previously revealed that AEB and LAD were excellent predictors of AF in ESUS patients.¹³⁾ When combining with the DLA, the negative predictive value was 0.974 (LAD <38.5 mm, AEB <0.050%, and DLA <0.5), which could be sufficient for use as a gatekeeper in AF screening in ESUS patients.

Specific morphological changes of ECG were proposed by using a ShapeExplainer, which is a generative adversarial network-based method for the interpretation of DLA classification model. In previous ECG studies using deep learning, GradCAM was applied for model interpretation.²¹⁾ However, GradCAM cannot present a specific morphological change and basis. A specific and visual model interpretation was provided through ShapeExplainer, which revealed that the prolonged PR and QT interval, and increased QRS amplitude were some of the decision criteria for identifying paroxysmal AF in SR ECG. The association between prolonged PR or QT interval and AF risk were proven by Atherosclerosis Risk In Communities cohort study, which including >15,000 participants,²²⁾,²³⁾ and a meta-analysis reported that left ventricular hypertrophy, in which QRS amplitude increases and is strongly associated with AF.²⁴⁾ The decision criteria of the DLA were consistent with results of previous studies and quantitative analysis of DLA could suggest additional knowledge for understanding heart electrophysiology.

A systematic review of ESUS revealed that it accounts for up to 25% of all ischemic strokes, and these patients have a 4.5%/year of the stroke recurrence rate.²⁵⁾ Of all ESUS patients, 30% patients have AF-related stroke and anticoagulation can effectively prevent stroke recurrence in these patients.²⁾ Most AF associated with ESUS exhibit paroxysmal features, which renders diagnosis difficult. Although several studies can increase the AF diagnosis rate through enhanced and prolonged Holter electrogram monitoring or find more paroxysmal AF using ICM in ESUS patients,²⁶⁾ their application to many patients is limited because of the invasive nature of these test. The prediction models using clinical risk factors have been developed to identify paroxysmal AF, such as CHARGE-AF, C2HEST, and HATCH.⁶⁾,¹⁴⁾,¹⁵⁾ These models revealed modest diagnostic performance in original studies; however, these studies did not achieve satisfactory diagnostic performance in external validation studies.²⁷⁾,²⁸⁾,²⁹⁾

Since Attia et al.¹⁰⁾ published a deep learning model that predicts the occurrence of AF during SR ECG, several models using the convolutional neural network architecture were proposed to predict AF, and these models exhibited excellent diagnostic performance as AUC 0.8–0.9.³⁰⁾ However, theoretically, these models may not provide very precise results because patients with undiagnosed paroxysmal AF could be misclassified to the control group during the development and validation process. Our DLA also has the same intrinsic limitation on the development process. However, the most accurate validation process could be performed using ICM in present study. The prevalence of AF in our database was found to be 27%. This low prevalence of the target condition can explain the low positive predictive value (PPV) and AUPRC, despite achieving a high AUROC of 0.862 in internal validation. Specifically, when the prevalence is low, the proportion of false positives can be relatively high, which can lower the PPV, and hence the AUPRC. Therefore, achieving a high AUPRC and PPV in low-prevalence settings can be challenging, and the performance of a predictive model in such scenarios should be evaluated in the context of the underlying prevalence rate. However, the fact that the PPV of the model is more than twice as high as the prevalence indicates that it is identifying meaningful characteristics of true positives.

This study has several limitations. First, this study is a retrospective study. Although ICMs are capable of detecting every single AF episode, the causality between the AF and the stroke still remains unclear. Second, this study was based on Korean patients only and the number of patients who underwent ICM was small, so our research might have insufficient generalizability. Third, ESUS is a complex condition with various sources beyond AF. The multi-variable model in this study might function differently in the general population. Even though AEB and LA size are well-established AF risk factors, our study did not validate their combined effect, which is a limitation in evaluating our model’s performance. Fourth, the potential influence of comorbidities, medication use, and other factors such as AF ablation and the use of antiarrhythmic agents on ECG features, which could introduce confounding variables affecting the configuration of SR ECGs. There is some concern that changes in the SR ECG may be due to these factors rather than to the pathology of AF, but several large studies in the past have provided evidence of ECG changes in patients with paroxysmal AF. Finally, we developed explainable AI models that can considerably expand our understanding of the algorithm. We developed ShapeExplainer to determine the region of interest of the ECGs for AF prediction. However, we could not provide a quantitative pattern of algorithm for decision making. In a future study, the black box of the DLA will be examined using emerging technologies.

The DLA accurately identified paroxysmal AF using 12-lead SR ECG in patients with ESUS and outperformed conventional models. By incorporating atrial ectopic burden (AEB) and left atrial diameter (LAD) into the DLA, the combined prediction model exhibited the highest level of diagnostic accuracy. Consequently, this model holds promise for identifying paroxysmal AF in patients with ESUS.

SUPPLEMENTARY MATERIALS

Supplementary Data 1

Variables and formula for conventional atrial fibrillation prediction models.

Supplementary Table 1

Baseline characteristics of patients for deep learning algorithm development in hospital A and B

Supplementary Table 2

Performances of developed DLA for detecting atrial fibrillation during sinus rhythm

Supplementary Table 3

Contingency matrix of DLA

Supplementary Figure 1

Receiver operating characteristic curves for DLA development and internal validation.