Artificial intelligence in detecting left atrial appendage thrombus by transthoracic echocardiography and clinical features: the Left Atrial Thrombus on Transoesophageal Echocardiography (LATTEE) registry

Abstract

Aims

Transoesophageal echocardiography (TOE) is often performed before catheter ablation or cardioversion to rule out the presence of left atrial appendage thrombus (LAT) in patients on chronic oral anticoagulation (OAC), despite associated discomfort. A machine learning model [LAT-artificial intelligence (AI)] was developed to predict the presence of LAT based on clinical and transthoracic echocardiography (TTE) features.

Methods and results

Data from a 13-site prospective registry of patients who underwent TOE before cardioversion or catheter ablation were used. LAT-AI was trained to predict LAT using data from 12 sites (n = 2827) and tested externally in patients on chronic OAC from two sites (n = 1284). Areas under the receiver operating characteristic curve (AUC) of LAT-AI were compared with that of left ventricular ejection fraction (LVEF) and CHA₂DS₂-VASc score. A decision threshold allowing for a 99% negative predictive value was defined in the development cohort. A protocol where TOE in patients on chronic OAC is performed depending on the LAT-AI score was validated in the external cohort. In the external testing cohort, LAT was found in 5.5% of patients. LAT-AI achieved an AUC of 0.85 [95% confidence interval (CI): 0.82–0.89], outperforming LVEF (0.81, 95% CI 0.76–0.86, P < .0001) and CHA₂DS₂-VASc score (0.69, 95% CI: 0.63–0.7, P < .0001) in the entire external cohort. Based on the proposed protocol, 40% of patients on chronic OAC from the external cohort would safely avoid TOE.

Conclusion

LAT-AI allows accurate prediction of LAT. A LAT-AI-based protocol could be used to guide the decision to perform TOE despite chronic OAC.

Structured Graphical Abstract

Structured Graphical Abstract.

Development and validation of an artificial intelligence model (LAT-AI) to detect left atrial appendage thrombus by transoesophageal echocardiography. AI, artificial intelligence; LAT, left atrial appendage thrombus; LV, left ventricular; CHA₂DS₂-VASc, current clinical score to assess thrombo-embolic risk.

See the editorial comment for this article ‘Artificial intelligence and innovation of clinical care: the need for evidence in the real world’, by A.J. Fletcher et al ., https://doi.org/10.1093/eurheartj/ehad553.

Introduction

Atrial fibrillation (AF) and atrial flutter promote the formation of left atrial appendage thrombus (LAT)¹ and are associated with an increased risk of thrombo-embolic events.² Although the mechanism of stroke is complex, a LAT has often been identified as the source of thrombus formation in patients who have recently experienced strokes.³ The restoration of sinus rhythm is associated with an increased risk of thrombo-embolic events,⁴ and embolization of already existing thrombi present in the atrium is considered as one of the possible causes.⁵ For this reason, cardioversion and catheter ablation in the presence of LAT are contraindicated.^6,7

Oral anticoagulation (OAC) reduces the risk of the formation of LAT but does not abolish this risk completely.^1,8–10 Transoesophageal echocardiography (TOE) is therefore considered the modality of choice to detect LAT with high sensitivity and specificity and is currently recommended^6,7,11 before cardioversion or catheter ablation as an alternative to a 3-week course of OAC. While not compulsory in chronically anticoagulated patients,^7,11 TOE is still performed routinely before catheter ablation or cardioversion in many centres.¹² However, TOE is associated with significant discomfort for the patient and may lead to complications.¹³ It is a more complex procedure which is time-consuming, requires more expensive equipment, resulting in higher costs¹⁴ compared to transthoracic echocardiography (TTE). Therefore, there is an unmet clinical need for an improved assessment of the risk of LAT.

TTE is relatively much faster and easier to perform than TOE. Several registries identified features detectable by TTE that are associated with the presence of LAT,^1,8–10 but none of the prior studies attempted to integrate the clinical and TTE data to provide a personalized assessment of the risk of LAT. This study aimed to develop and validate an accessible and practical for clinical use method to predict LAT with high sensitivity using readily available clinical features and TTE measurements. To achieve this, we utilized artificial intelligence (AI) and tested the method in an independent external cohort. Further, we sought to investigate the clinical applicability of such a tool through simulated cancellation of TOE studies.

Methods

Study data

We used data from the multi-site prospective Left Atrial Thrombus on Transoesophageal Echocardiography (LATTEE) registry^10,15 (ClinicalTrials.gov identifier: NCT03591627) that included 3227 patients undergoing TOE before cardioversion or catheter ablation at 13 sites, as well as 1075 cases collected retrospectively across two sites, resulting in a total number of 4302 cases and 14 sites in the study. The decision on whether to perform TOE before catheter ablation or cardioversion was done at the discretion of the attending physicians following site-specific procedures.¹⁰ All TOE studies were performed by echocardiographers certified in TOE by the Polish Society of Echocardiography and independently reviewed by a second echocardiographer at the study site. Data from 12 sites (n = 2827) served as the development cohort and were used for internal validation. Data from the single site that recruited the largest number of patients, as well as the two retrospective cohorts, were set aside as an external testing set (N = 1475, two different sites). Patients who did not receive OAC for at least 3 weeks before TOE were excluded from the external testing set (n = 191), resulting in a final size of 1284 patients. More details about the dataset are given in a consort diagram (see Supplementary data online, Figure S1). The CHA₂DS₂-VASc score (congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, history of stroke or thrombo-embolism, vascular disease, age 65–74 years, female sex) was calculated using the standard definition.⁷

Compliance with the Declaration of Helsinki

This study complies with the Declaration of Helsinki. The institutional review boards at all participating sites approved the collection of data for the registry. This was an observational study, with no modification in patient management. Therefore, the requirement for written informed consent was waived.

Model and feature selection

As a first step, we performed a comprehensive model and feature selection using a large-scale AI evaluation framework (STREAMLINE¹⁶) to choose the best machine learning model type, model hyperparameters, and an optimal set of features. Using this tool, we evaluated 13 different models, including logistic regression (LR), tree-based models, and neural networks in the development cohort (see also Supplementary data online, Methods and Supplementary data online, Figure S2).

Training and internal testing

In the next step, we implemented the machine learning classifier for the prediction of LAT (LAT-AI) directly in Python programming language using the optimal hyperparameters established in the previous step. The full list of features (28 clinical, 7 blood test results, and 4 TTE features) used by LAT-AI is given in Supplementary data online, Table S1. We first trained and tested LAT-AI internally using a 10-fold cross-validation regimen, which is currently the technique of choice in data mining.¹⁷ We used average absolute Shapley Additive Explanations (SHAP values)¹⁸ as a measure of feature importance in the development cohort.

Model with reduced number of features

In order to facilitate the practical application of our model, we developed a simplified model (LAT-AI-reduced) with a reduced number of features. We used a previously developed method¹⁹ to select a minimal set of features that retains 99% of the predictive performance of the full model, based on an internal 10-fold cross-validation. To demonstrate the potential clinical use of the developed approach, we show how the LAT-AI-reduced model could be used as an easy-to-use web application. This is described in greater detail in the Supplementary data online, Methods.

Final models

In the final step, we retrained the LAT-AI and LAT-AI-reduced models in the entire development cohort using the same settings as in the 10-fold cross-validation. Additionally, to demonstrate how our models compare to an optimal simple LR model, we selected two to four most contributing features as input parameters for the LR models and systematically evaluated multiple LR models using these sets of features with an optional addition of the CHA₂DS₂-VASc score in the development cohort using 10-fold cross-validation. Subsequently, the best-performing LR model was evaluated using the external testing set.

External testing

We applied the final LAT-AI and LAT-AI-reduced models, as well as two simple LR models, to the external testing cohort. For a given patient, LAT-AI and LAT-AI-reduced models generated their prediction as continuous score values. For external testing, missing values were not imputed but passed to the model ‘as is’.

Simulated cancellation of TOE

We identified the decision thresholds for the simulated cancellation of TOE in patients on chronic OAC based on the results from the development cohort. The thresholds were defined separately for the LAT-AI, LAT-AI reduced, the best LR model, and left ventricular ejection fraction (LVEF) as the highest score value (or lowest LVEF value), allowing for at least 99% negative predictive value for LAT. We then applied these thresholds to the external dataset by assuming the cancellation of TOE in all patients who had predicted score lower than the threshold. Subsequently, we evaluated the negative predictive value, sensitivity, and specificity in unseen data, as well as the number of TOE studies that could potentially be avoided using LAT-AI, LAT-AI-reduced, and LR models.

Patient-level explanation

At the time of inference in the external testing set, per-patient SHAP values were generated and visualized using waterfall plots that showcase the degree and direction of the contribution of each feature for an individual patient-level prediction.

Compliance with recommendations for machine learning-related research

This study was designed and conducted following recently published guidance papers on the use of AI-based predictive models in cardiovascular research.^20,21 To improve the transparency of reporting and the reproducibility of machine learning algorithms, the Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation checklist is included in Supplementary data online, Table S2.

Statistical analysis

Continuous variables were expressed by median and interquartile ranges (IQRs). Median values were compared with a two-sided Wilcoxon rank-sum test or a Kruskal–Wallis test. Categorical variables were compared using Fisher’s exact test.

We used area under receiver operating characteristic curves (AUCs) to compare the classification performance of LAT-AI and LAT-AI-reduced models and compare them against known predictors of LAT and against simple LR models as well as to compare the predictive performance depending on sex. The performance of the models, as well as the ability to cancel TOE, was evaluated in patients who received chronic OAC. DeLong’s method²² was used for comparisons between AUC values as well as to obtain 95% confidence intervals (CIs) for AUC. Power of statistical comparisons of AUC was assessed using pROC package.²³ Specificity, sensitivity, and negative predictive value of LAT-AI and LAT-AI-reduced models were calculated for the decision threshold for the simulated cancellation of TOE, and 95% CIs were generated using the exact binomial method.²⁴ Sensitivity of the models and individual predictors was compared using the McNemar test. Calibration of the models was assessed using plots of observed event rates vs. predicted scores and using calibration slopes with 95% CI in the external testing set. We used multivariable LR analysis to evaluate the independent diagnostic performance of LAT-AI and LAT-AI-reduced models after adjusting for other strong predictors of LAT (LVEF, left atrial dimension, and non-paroxysmal arrhythmia). We also used decision curves²⁵ to evaluate the net benefit of the use of LAT-AI and LAT-AI-reduced models compared to non-paroxysmal arrhythmia and reduced LVEF. A P-value <.05 was considered significant.

All the machine learning operations were performed in Python 3.8.13, while statistical analysis was performed using R 4.1.2 and RStudio software. A detailed listing of all used packages and versions is given in Supplementary data online, Table S3.

Results

Development cohort characteristics

Detailed characteristics of the development cohort are presented in Table 1. The median age of patients was 67 (IQR: 59–73) years, and 37% of them were female. On TOE, LAT was detected in 224 patients (7.9%); 85% of them (n = 209) had persistent arrhythmia and 77% (n = 173) received chronic OAC. Patients without chronic OAC had thrombus more frequently (12%) than patients on chronic OAC (7.2%, P < .001). Although half of the patients were planned for catheter ablation, the majority of LAT (82%) were detected in patients undergoing cardioversion.

Table 1.

Development cohort characteristics

Characteristic	Overall, N = 2827	No LAT, N = 2603 (92.1%)	LAT N = 224 (7.9%)	P-value
Age, years	67 (59–73)	66 (59–73)	70 (63–78)	<.001
Female sex	1048 (37%)	971 (37%)	77 (34%)	.4
Cardioversion (vs. ablation)	1401 (50%)	1218 (48%)	183 (82%)	<.001
Atrial fibrillation	2495 (88%)	2300 (88%)	195 (87%)	.5
Atrial flutter	408 (14%)	375 (14%)	33 (15%)	.9
Paroxysmal (vs. persistent)	1181 (42%)	1148 (44%)	33 (15%)	<.001
BMI, kg/m2	29.0 (26.0–33.0)	29.2 (26.0–33.0)	29.0 (26.0–32.0)	.13
NYHA class				<.001
I–II	836 (30%)	747 (29%)	89 (40%)
III	252 (8.9%)	202 (7.8%)	50 (22%)
IV	34 (1.2%)	28 (1.1%)	6 (2.7%)
No HF	1705 (60%)	1626 (62%)	79 (35%)
Mitral stenosis				<.001
Mild	13 (0.5%)	9 (0.3%)	4 (1.8%)
Moderate to severe	14 (0.5%)	9 (0.3%)	5 (2.2%)
History of stroke				.048
Haemorrhagic stroke	14 (0.5%)	11 (0.4%)	3 (1.3%)
Ischaemic stroke	175 (6.2%)	156 (6.0%)	19 (8.5%)
No history of stroke	2638 (93%)	2436 (94%)	202 (90%)
Chronic kidney disease	370 (13%)	326 (13%)	44 (20%)	.002
Hypertension	1862 (75%)	1705 (75%)	157 (76%)	.7
Diabetes mellitus	702 (25%)	619 (24%)	83 (37%)	<.001
Labile INR	68 (2.6%)	50 (2.1%)	18 (8.5%)	<.001
Alcohol overuse	122 (4.4%)	100 (3.9%)	22 (10%)	<.001
Smoking				<.001
Currently	289 (10%)	259 (10.0%)	30 (13%)
In the past	682 (24%)	609 (23%)	73 (33%)
Non-smoker	1856 (66%)	1735 (67%)	121 (54%)
Creatinine	1.00 (0.90–1.20)	1.00 (0.90–1.20)	1.10 (0.90–1.21)	.008
Left atrial AP dimension, mm	45 (41–50)	45 (41–49)	48 (44–52)	<.001
LVEF, %	55 (45–60)	55 (46–60)	42 (30–51)	<.001
Left atrial area, cm2	26 (22–30)	26 (22–30)	28 (25–33)	<.001
Left atrial volume index, mL/m2	45 (36–56)	45 (36–55)	53 (43–63)	<.001
Rhythm at the time of study				<.001
Sinus rhythm	778 (28%)	766 (30%)	12 (5.4%)
Atrial fibrillation	1805 (64%)	1612 (62%)	193 (87%)
Atrial flutter	227 (8.1%)	211 (8.1%)	16 (7.2%)
On chronic OAC	2394 (85%)	2221 (86%)	173 (77%)	<.001
CHA2DS2-VASc score	3 (1–4)	3 (1–4)	4 (3–5)	<.001

Statistics presented: median (interquartile range) and n (%). The bold values represent statistically significant (P < .05) comparisons.

AP, anteroposterior; BMI, body mass index; HF, heart failure; INR, international normalized ratio; LA, left atrium; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association; OAC, oral anticoagulation.

External cohort characteristics

Detailed characteristics of the external testing cohort are presented in Table 2. The median age of patients was 67 (IQR: 59–73) years, 37% of them were female, and 495 (43%) had paroxysmal arrhythmia. LAT was found in TOE in 71 cases (5.5%), and 6.2% of patients with LAT had paroxysmal arrhythmia. Cardioversion was planned in 50% of patients (n = 645), and LAT was found more frequently in patients planned for cardioversion than for catheter ablation (9.8% vs. 1.3%, P < .001). Counts of missing values for the development and external testing cohorts are given in the Supplementary data online, Table S4.

Table 2.

External cohort characteristics

Characteristics	Overall, N = 1284	No LAT, N = 1213 (94.5%)	LAT, N = 71 (5.5%)	P-value
Age, years	67 (59–73)	67 (59–73)	71 (62–77)	.004
Female sex	471 (37%)	450 (37%)	21 (30%)	.2
Cardioversion (vs. ablation)	645 (50%)	582 (48%)	63 (89%)	<.001
Atrial fibrillation	1112 (87%)	1050 (87%)	62 (87%)	.9
Atrial flutter	182 (14%)	173 (14%)	9 (13%)	.7
Paroxysmal (vs. persistent)	495 (43%)	491 (45%)	4 (6.2%)	.001
BMI, kg/m2	29.0 (26.0–32.0)	29.0 (26.0–32.4)	27.8 (24.7–30.0)	.011
NYHA class				<.001
I–II	426 (33%)	394 (33%)	32 (45%)
III	177 (14%)	143 (12%)	34 (48%)
IV	7 (0.5%)	6 (0.5%)	1 (1.4%)
No HF	672 (52%)	668 (55%)	4 (5.6%)
Mitral stenosis				.6
Mild	8 (0.7%)	7 (0.6%)	1 (1.4%)
Moderate to severe	8 (0.7%)	8 (0.7%)	0 (0%)
History of stroke				.4
Ischaemic stroke	74 (6.1%)	68 (6.0%)	6 (8.6%)
No history of stroke	1138 (94%)	1074 (94%)	64 (91%)
Chronic kidney disease	358 (28%)	311 (26%)	47 (66%)	<.001
Hypertension	988 (78%)	926 (77%)	62 (87%)	.042
Diabetes mellitus	312 (24%)	287 (24%)	25 (35%)	.030
Labile INR	8 (0.7%)	4 (0.4%)	4 (5.7%)	<.001
Alcohol overuse	24 (2.0%)	18 (1.6%)	6 (8.6%)	.002
Smoking				<.001
Currently	125 (9.8%)	107 (8.9%)	18 (25%)
In the past	213 (17%)	196 (16%)	17 (24%)
Non-smoker	937 (73%)	901 (75%)	36 (51%)
Left atrial AP dimension, mm	45 (41–50)	45 (41–49)	50 (47–54)	<.001
LVEF, %	56 (45–62)	57 (45–62)	33 (27–45)	<.001
Left atrial area, cm2	27 (23–32)	27 (23–31)	33 (30–36)	<.001
Left atrial volume index, mL/m2	43 (35–53)	42 (34–52)	60 (53–69)	<.001
Rhythm at the time of study				<.001
Sinus rhythm	362 (28%)	361 (30%)	1 (1.4%)
Atrial fibrillation	789 (61%)	726 (60%)	63 (89%)
Atrial flutter	133 (10%)	126 (10%)	7 (9.9%)
CHA2DS2-VASc score	3 (2–4)	3 (2–4)	4 (3–5)	<.001

Statistics presented: median (inter quartile range) and n (%). The bold values represent statistically significant (P < .05) comparisons.

AP, anteroposterior; BMI, body mass index; HF, heart failure; INR, international normalized ratio; LA, left atrium; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association.

Training and internal testing

The Extreme Gradient Boosting model²⁶ was proven to provide the best predictive performance in the STREAMLINE analysis (see Supplementary data online, Figure S2) and was therefore used for subsequent training of LAT-AI. The most important features driving the predictions of LAT-AI in the development cohort were left atrial measurements [especially left atrial volume index (LAVI)] and the presence of heart failure (Figure 1A).

Figure 1.

Average feature importance scores in the order of importance with 95% confidence intervals (whiskers) based on the internal 10-fold cross-validation in the development cohort. A, For the full LAT-AI model (top 20 features); B, for the LAT-AI-reduced model. AP, anteroposterior; INR, international normalized ratio; NYHA, New York Heart Association.

Reduction of the number of features

The minimal number of features that allowed for 99% of the predictive performance of the full model in the internal, 10-fold cross-validation using the development cohort was 8: age, arrhythmia duration, rhythm at the time of TOE, heart failure, New York Heart Association functional class, LVEF, left atrial anteroposterior dimension, and LAVI (Figure 1B).

Definition of decision thresholds for TOE cancellation

The decision threshold for cancellation of TOE was 0.37 according to the LAT-AI model and had a 99% negative predictive value (95% CI: 98–99), 38% specificity (95% CI: 36–40), and 93% sensitivity (95% CI: 89–96) in patients from the development cohort who received chronic OAC. For the LAT-AI-reduced model, the decision threshold was found to be 0.34, allowing for 99% negative predictive value (95% CI: 97–100), 26% specificity (95% CI: 24–28), and 97% sensitivity (95% CI: 95–100) in patients from the development cohort who received chronic OAC. For the best LR model, the decision threshold allowing for at least 99% negative predictive value (95% CI: 98–99) in the development cohort was found to be 0.05 and achieved 92% sensitivity (95% CI: 88–96) and 44% specificity (95% CI: 42–46). For LVEF, the threshold allowing for at least 99% negative predictive value in the development cohort was 65% and achieved 100% sensitivity (95% CI: 0.97–100) and 3% specificity (95% CI: 2–4) in the development cohort.

External testing

In the external testing set of 1284 patients on chronic OAC undergoing cardioversion or catheter ablation, LAT-AI outperformed the CHA₂DS₂-VASc score as well as the most contributing feature, LVEF (Figure 2). Both LAT-AI (AUC 0.85, 95% CI: 0.82–0.89) and LAT-AI-reduced (AUC 0.84, 95% CI: 0.8–0.89) models performed also significantly better than other individual predictors of LAT such as LVEF (AUC 0.81, 95% CI: 0.76–0.86) or LAVI (AUC 0.81, 95% CI: 0.76–0.87, P = 0. 001) or arrhythmia duration (0.67, 95% CI: 0.63–0.71, P < .0001). The best set of features for the LR model, as established in the development cohort, consisted of LVEF, LAVI, rhythm at the time of the study, and CHA₂DS₂-VASc score (comparison of all candidate feature sets is provided in Supplementary data online, Table S5). The LR model trained using these parameters achieved an AUC of 0.83 (95% CI: 0.79–0.88), which was significantly worse than the LAT-AI model (P = .007) but not significantly different from the LAT-AI-reduced model (P = .25). The comparison of the models and individual features is provided in Supplementary data online, Table S6. The statistical power of comparisons of the AUC of our models and LVEF was estimated as 93% and 83% for the LAT-AI and LAT-AI-reduced models, respectively.

Figure 2.

Receiver-operating characteristic curves for the prediction of left atrial thrombus in the external testing set. Significance for difference in AUC (by DeLong test): *P < .001; **P < .01. AUC, area under the receiver-operating characteristic curve; LAT, left atrial appendage thrombus; LVEF, left ventricular ejection fraction; ML, machine learning model; NS, non-significant.

The LAT-AI score remained the strongest predictor of LAT after adjusting for the input parameters of the best LR model (LVEF, LAVI, rhythm at the time of study, and the CHA₂DS₂-VASc score) [odds ratio (OR): 12.2, 95% CI: 3.6–47.3]. Similarly, the LAT-AI-reduced score remained the only predictor significantly associated with LAT after adjusting for the same factors (OR 8, 95% CI: 2.8–25.4), as shown in Supplementary data online, Table S7.

The AUC of the LAT AI and LAT-AI-reduced models in men and women did not differ significantly but were higher for women, as shown in Supplementary data online, Table S8. Both LAT-AI and LAT-AI-reduced models provided benefit over established predictors of LAT across a wide range of risk thresholds (see Supplementary data online, Figure S3). Calibration of the models is presented in Supplementary data online, Figure S4. Two examples of patient-level explanation of the model’s prediction in the external testing set are shown in Supplementary data online, Figure S5.

Simulated application of management based on LAT-AI

The application of LAT-AI to guide the decision on whether to perform TOE before cardioversion or ablation was performed in the external testing set (n = 1284). By not performing TOE in patients with LAT-AI score < 0.37, it would be possible to avoid 40% of routine TOE (n = 512) studies with 100% negative predictive value (95% CI: 99–100), 100% sensitivity (95% CI: 95–100), and 42% specificity (95% CI: 39–45) in this group (Figure 3). Using the LAT-AI-reduced score and threshold of 0.34, it would be possible to avoid 26% of routine TOE (n = 328) studies with 100% negative predictive value (95% CI: 98–100), 100% sensitivity (95% CI: 95–100), and 27% specificity (95% CI: 25–30). A screenshot of the prototype web application for the LAT-AI-reduced model is shown in Supplementary data online, Figure S6. The best LR model with the threshold of 0.05 would mandate the cancellation of 48% of studies (n = 620) but with five false negative predictions (resulting in significantly lower sensitivity of 93%, 95% CI: 0.84–0.98) than that of LAT-AI models (P = .03). By cancelling TOE based on LVEF >65%, it would be possible to exclude all LAT in the external testing set (sensitivity of 100%, 95% CI: 0.95–100), but such an approach would only allow avoiding 6.3% of TOE studies (n = 82).

Figure 3.

Simulated application of LAT-AI and LAT-AI-reduced models to guide the decision to perform TOE in the external cohort, based on the thresholds derived from the development cohort. LAT, left atrial appendage thrombus; TOE, transoesophageal echocardiography; OAC, oral anticoagulation.

Discussion

We developed and externally validated a novel approach for the prediction of LAT in patients with AF or atrial flutter. We demonstrate that through the integration of clinical and TTE features, it is possible to achieve a far superior predictive performance for the detection of LAT compared to any other marker of thrombo-embolic risk and that such a score could be used to select patients who should undergo TOE before cardioversion or catheter ablation. Our approach could allow up to 40% of patients on chronic OAC to avoid the discomfort, potential risks, cost, and time associated with TOE. Finally, it could prevent serious thrombo-embolic complications by mandating TOE in high-risk patients (despite adequate chronic OAC), in whom, following the current guidelines,⁷ TOE would not be performed (Structured Graphical Abstract).

According to a European multi-centre registry, pre-ablation TOE was performed in up to 91.3% of patients undergoing catheter ablation,¹² but a recent meta-analysis showed that the incidence of LAT is <2% in patients undergoing catheter ablation and 8.1% in patients undergoing cardioversion⁸ . Current guidelines leave room for individual decisions regarding the necessity of performing a TOE before catheter ablation or cardioversion in patients on chronic OAC, but at the same time offer no tools for the assessment of the pre-test likelihood of LAT.^7,27 Milhem et al.²⁸ proposed an approach to exclude LAT using a simple four-feature score, but this method was not externally validated. The Heart Rhythm Society’s recommendations for management before catheter ablation define TOE in patients on chronic OAC and in AF on presentation as reasonable (Class IIa recommendation),¹¹ and in everyday clinical practice routine pre-ablation TOE is performed in many centres.¹² Similarly, the guidelines of the European Heart Rhythm Association state that it remains an individualized decision whether to perform a pre-cardioversion TOE for thrombus exclusion, even when subjects are effectively anticoagulated.²⁷ That leaves a vast majority of patients undergoing cardioversion or catheter ablation in a ‘grey zone’—meaning that either performing TOE to exclude LAT or proceeding without TOE would be an acceptable management. In this study, we target this ‘grey zone’ and provide a support tool for the physician to assess the risk of LAT and guide the decision to perform TOE.

Previous studies identified multiple highly informative features that would broadly stratify patients into groups of high and low risk of LAT.^1,8,9,29,30 These include reduced LVEF^9,31 or left atrial dilation^31,32 that can be easily and quickly obtained with TTE. Melduni et al.⁹ showed in a large single centre cohort that LVEF is the best single predictor of the presence of LAT with AUC of 0.78, and the LVEF ≤ 40% had a sensitivity of 62% and specificity of 75% in the detection of LAT. Notably, our method is characterized by much higher sensitivity than reduced LVEF alone, and while our data confirm left ventricular systolic function as being the top predictor of LAT, 31% of patients who had thrombi in our external dataset had a LVEF > 40%. Another study suggested that increased left atrial volume could be the hallmark of an elevated risk of LAT.³³ There is a growing body of evidence that the standard TTE assessment of both left ventricle and atrium contributes valuable information and allows for better stratification of the risk of LAT than clinical features alone. Our method, however, is the first to combine these multiple predictors with clinical data into a single, highly sensitive score, developed with state-of-the-art AI techniques and validated in multi-site external cohort.

Performing TOE routinely in all patients undergoing catheter ablation or cardioversion is not cost-effective¹⁴ and would put patients at an unnecessary risk of complications that include oropharyngeal injuries, gastro-oesophageal trauma that may result in perforation or bleeding, respiratory complications, and dysphagia.¹³ On the other hand, two studies suggested that catheter ablation could be safely performed without prior TOE in patients on chronic non-vitamin K antagonist OAC.^34,35 However, these were single-site studies where the low risk of LAT might have been affected by a selection bias. These results might therefore only be applicable to this specific population, as a European registry revealed substantial differences between sites in terms of patient and procedural profiles.³⁶ Moreover, as catheter ablation of AF is currently recommended to reverse severe left ventricular dysfunction,⁷ which is also a risk factor for LAT,⁹ it can be expected that the incidence of LAT in populations of patients undergoing catheter ablation will increase. We address the heterogeneity of patients undergoing catheter ablation or cardioversion by a robust AI model that was developed in a diverse multi-centre population and integrates a wide variety of information including patient history, heart failure status, and LVEF.

We observed that the LAT-AI and LAT-AI-reduced models had the highest AUC values among all the models and individual predictors evaluated in external testing. While the AUC of the LAT-AI-reduced model was not significantly different from that of the best LR model, the sensitivity of the LAT-AI-reduced model was higher. Considering clinical safety, high sensitivity is a significant advantage. Notably, the best LR model utilized LVEF, LAVI, rhythm at the time of the study, and the CHA₂DS₂-VASc score (which aggregates 7 features). Therefore, the best LR model effectively requires 10 individual parameters—more than the LAT-AI-reduced model, which requires only eight (see Supplementary data online, Figure S6).

Moreover, the LAT-AI and LAT-AI-reduced models provide an integrated visual explanation of predictions generated at a patient level. Visual depiction of features contributing to the high (or low) risk of LAT (see Supplementary data online, Figure S5) may increase the trust in the model and facilitate its adoption.^37–39 Furthermore, the individualized explanation may allow for a more comprehensive assessment of the patient’s condition and integrate the multitude of clinical data into a single visual summary.

Study limitations

Our study utilized conservative two-site external validation with 1284 cases but for such rare events as LAT, even larger testing sets might be recommended.⁴⁰ However, to our knowledge, this is the first study to develop and validate in a multi-site external cohort a prediction model for LAT. Future studies in different populations are needed to assess the performance of our models in various settings, as well as to evaluate performance in specific sub-groups based on sex or race.

The prevalence of LAT in both development and validation cohorts was higher in this study than in previous publications.^8,41,42 The LATTEE registry was an observational study where patients were referred for TOE following local, site-specific policies, which are subject to variability.⁴³ Individuals assessed as having very low risk of LAT by the attending physician were not referred for pre-cardioversion and pre-ablation TOE, resulting in selection bias.

The full LAT-AI model requires multiple features, and considerable time is necessary to manually enter these features into the model. This limitation can, at least to some extent, be circumvented by taking advantage of data stored in electronic health records and integrating the LAT-AI model into the existing systems, so that data that are already present in a patient’s record do not require manual re-entering. In contrast, the LAT-AI-reduced model would not require blood test results and could be implemented as a simple web application (as demonstrated by our prototype) that may facilitate rapid clinical adoption and could be used instantaneously at the patient’s bedside.

Importantly, the developed models are intended to offer a single decision threshold that allow for proceeding without TOE while maintaining a high sensitivity for event detection. However, they are not designed to provide an actual probability of LAT in the overall population of patients who undergo catheter ablation or cardioversion.

While TTE is significantly faster, safer, and easier to perform compared to TOE, it still requires expertise and dedicated time. Application of deep neural networks for AI-based analysis of TTE recordings might allow for a more automated assessment of the risk of LAT with echocardiographic results derived by the AI system, but the deployment of image-based solution may prove more difficult than of LAT-AI or LAT-AI-reduced models.

Conclusions

We propose a tool for personalized and explainable predictions of the risk of LAT, which can guide decisions on whether to perform TOE before catheter ablation or cardioversion. This tool can help reduce the number of pre-ablation and pre-cardioversion TOE performed in patients on chronic OAC.

Supplementary data

Supplementary data are available at European Heart Journal online.

Declarations

Disclosure of Interest

Unrelated to this work, Piotr Slomka received research grants form Siemens Healthlineers, and consulting fees from Synektik, SA. Agnieszka Kapłon-Cieślicka received consulting fees from Angelini Pharma, Astra Zeneca, Bausch Health, KRKA, Polpharma and Servier. Ludmiła Daniłowicz-Szymanowicz, Agnieszka Kapłon-Cieślicka and Beata Wożakowska-Kapłon received consulting fees from Bayer. Ludmiła Daniłowicz-Szymanowicz, Agnieszka Kapłon-Cieślicka, Paweł Krzesiński, Anna Tomaszuk-Kazberuk and Beata Wożakowska-Kapłon received consulting fees from Boehringer Ingelheim. Rafał Wolny received consulting fees from Boston Scientific. Ewa Kowalik received consulting fees from GE Healthcare. Ludmiła Daniłowicz-Szymanowicz, Agnieszka Kapłon-Cieślicka, Anna Tomaszuk-Kazberuk and Beata Wożakowska-Kapłon received consulting fees from Pfizer.

Data and Code Availability

The code used for model training and inference, as well as a trained model, is available through the GitHub platform (https://github.com/konradpieszko/LAT-AI). The data used to generate these results are subject to data sharing agreements and cannot be made publicly available but can be shared following the agreement of respective institutional review boards on request to the corresponding author.

Funding

P.J.S. was supported by research grant R35-HL161195 from the National Heart, Lung, and Blood Institute at the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. J.K. is supported by the National Science Centre, Poland—2021/41/B/NZ5/02630. R.W. is supported by the National Science Centre, Poland—2021/43/D/NZ5/02434. The remaining authors were funded internally by their respective institutions.

Ethical Approval

Ethical approval was not required.

Pre-registered Clinical Trial Number

NCT03591627.