Keep your fingers on the PULsE: artificial intelligence to guide atrial fibrillation screening

This editorial refers to ‘Identification of undiagnosed atrial fibrillation using a machine learning risk-prediction algorithm and diagnostic testing (PULsE-AI) in primary care: a multi-centre randomized controlled trial in England', by N.R. Hill et al., https://doi.org/10.1093/ehjdh/ztac009.

To see things in the seed, that is genius.

Lao-Tzu

Undiagnosed atrial fibrillation (AF) is an important cause of stroke.¹ AF screening may enable prompt detection of AF and initiation of oral anticoagulation (OAC) to prevent stroke.² The 2007 SAFE trial reported a roughly 50% increase in AF diagnosis with screening individuals aged ≥65 years using electrocardiography (ECG) with or without pulse palpation,³ resulting in a Class I recommendation from the European Society of Cardiology⁴ and the Cardiac Society of Australia and New Zealand⁵ for AF screening using ECG among individuals aged ≥65 years.

However, more recent studies suggest that mass screening may not be effective.^6,7 The efficiency of AF screening may be improved by AF risk estimation,⁸ which is feasible using clinical risk scores.⁹ However, such scores have had limited uptake due to complexity, modest predictive performance, and lack of automation.¹⁰ Machine learning, a form of artificial intelligence (AI) comprising a variety of models utilizing iterative adjustment to minimize prediction error,¹¹ has demonstrated promise in disease risk prediction and has potential to address many of these limitations. Beyond prediction and screening, AI can influence how we treat patients with AF and potentially impact outcomes.

This issue of European Heart Journal: Digital Health reports the results of prediction of undiagnosed atrial fibrillation using a machine learning algorithm (PULsE)-AI,¹² a multi-centre randomized controlled trial testing use of a neural network AI model to identify individuals at high AF risk, who were then targeted for screening using 12-lead ECG and serial one-lead handheld ECG (Figure 1). Across six general practices in England, 23 745 participants aged ≥30 years without known AF were randomly allocated to intervention (n = 11 849) and control (n = 11 896) arms. A total of 768 individuals in the intervention arm with high AI-predicted risk of AF were offered AF screening, of whom 256 (33.3%) accepted. Individuals in the control group received usual care. Over the 20-month study period (extended from 6 months due to the COVID-19 pandemic), the incidence of the primary endpoint of AF, atrial flutter, and fast atrial tachycardia was 5.63 and 4.93% among individuals at high predicted AF risk in the intervention and control arms, respectively, which was not a statistically significant difference [odds ratio (OR) 1.15, 95% CI 0.77–1.73, P = 0.486]. Among intervention participants who actually underwent screening, however, the rate of the primary endpoint was 9.41%, which was substantially greater than the control rate (OR 2.23, 95% CI 1.31–3.73, P = 0.003).

Figure 1.

Overview of artificial intelligence-guided atrial fibrillation screening. Depicted is a summary of artificial intelligence-guided atrial fibrillation screening, the overall concept assessed in the PULsE-artificial intelligence study. The top panel provides an overview of the concept of artificial intelligence-enabled atrial fibrillation risk estimation to guide screening, in which only individuals who are estimated to have elevated atrial fibrillation risk using artificial intelligence are offered the screening intervention. The bottom panel highlights several challenges associated with artificial intelligence-based atrial fibrillation screening, with potential solutions proposed below.

Discussion

PULsE-AI is an important demonstration of prospective randomized assessment of an AI model intended to guide clinical practice. Despite a recent proliferation of AI-based models to predict disease,¹¹ there has been comparably little integration of AI models into real-world clinical practice.^10,11 Valid concerns surrounding clinical AI models include the potential for overfitting (i.e. poorer performance in populations distinct from those in which models were derived), thereby limiting their generalizability. Beyond this there are also concerns regarding whether AI models may disrupt clinical workflows on account of complexity, computation time, or provider apprehension.¹⁰ To this end, randomized assessment is critical to establish whether prospective deployment of AI models is feasible, valid, and clinically effective. Only by meeting such a standard will AI models gain widespread acceptance in clinical practice.

In this case, despite successful implementation, AI-enabled screening did not meaningfully improve AF detection, with similar AF diagnosis rates between the intervention and controlarms. Nevertheless, the AF diagnosis rate was roughly two-fold higher among those randomized to intervention who ultimately underwent screening, when compared with controls. Although the latter per-protocol analysis is subject to selection bias and must be interpreted with caution, it highlights the reality that for any AI-based intervention to be effective, it must be broadly acceptable to its target population. To this end, only a third of individuals invited to screening ultimately adhered. The authors propose that unfamiliarity with contemporary technology (e.g. one-lead ECG) may have been an important barrier. Importantly, this highlights the risk of furthering disparities in care especially amongst the elderly, rural, and disenfranchised communities. However, given evidence suggesting that electronic risk management may improve outcomes in AF,¹³ we submit that a fully remote screening option may have improved participation, particularly given the challenges of the COVID-19 pandemic. Indeed, in the future both AF risk estimation and active screening may even be performed using the same mobile technology (Figure 1).

PULsE-AI also provides an important demonstration of the potential for risk estimation to improve the efficiency of AF screening. The AI algorithm the investigators deployed was generally accurate, as individuals predicted to have elevated AF risk (i.e. AF probability ≥7.4%, a threshold corresponding to 90% specificity in the algorithm’s derivation) had an AF diagnosis rate of approximately 5%, as compared to 0.6% among individuals not classified as high-risk. Although there is evidence for some miscalibration, since one would expect an AF diagnosis rate greater than 5% when using a risk threshold of 7.4%, such enrichment remains powerful when compared to recent AF screening trials which report AF incidence rates of roughly 1–2%/year among individuals aged ≥65 years (i.e. those with a guideline-based indication for AF screening^4,5). Future work is needed to establish whether AF risk estimation may be deployed more broadly to prioritize individuals for AF screening and whether such screening improves outcomes.

Although the work by Hill et al.¹² is an important advance, several important considerations remain. Notably, their algorithm utilizes a vast array of detailed clinical risk factor information to generate predictions. Recent models utilizing AI-enabled analysis of raw data (e.g. ECG^14,15) may reduce reliance on electronic health record-based clinical inputs which may be subject to misclassification. Beyond this, even very efficient AF detection may fail to improve outcomes if screen-detected AF is not truly actionable. Future work is needed to assess whether AI-based methods can be used to enrich for actionable AF and better integrate diagnosis with initiation of oral anticoagulationand other preventive interventions. Third, prospective evaluation is key, but AI-based risk models also require broad external validation in dissimilar populations (e.g. outside the UK) before generalizability can be established.

In summary, PULsE-AI provides demonstration of the feasibility of AI-based AF risk estimation and provides a good example of how to deploy and assess clinical AI-based interventions. Whether AI-enabled AF risk estimation can improve the efficiency of AF screening and lead to improved outcomes will need further investigation. It is quite possible that these calculated risks will yield considerable rewards.

Data availability

This editorial does not include any original data.