Going Deep With ECG and Aortic Stenosis: Touchdown or Incomplete Pass?

A little learning is a dangerous thing; Drink deep, or taste not the Pierian spring.

Alexander Pope

Aortic stenosis (AS) is a growing problem in aging Western populations.1 Many patients with AS are unaware of it and present late, often after irreversible left ventricular hypertrophy/fibrosis and/or diastolic or systolic dysfunction.2 Whereas surgical guidelines currently dictate that either open heart surgery or transaortic valve replacement are reserved for those with clear symptoms, emerging evidence suggests that intervention may be preferred in some populations with severe asymptomatic AS.3 With expanding indications and thresholds for transaortic valve replacement, screening and access to evidence‐based therapies are increasingly important. Furthermore, with limited access to echocardiography, the main imaging modality for AS, there is a need to identify alternative diagnostic modalities that are portable and community focused.

See Article by Kwon et al.

In this issue of the Journal of the American Heart Association (JAHA), Kwon et al. took a first step in evaluating a deep learning (DL) algorithm with a view toward detecting moderate‐to‐severe AS via ECG.4 Their optimized DL approach employed both a multilayer perceptron and a convolutional neural network as an ensemble to interrogate patterns within 12‐lead and single‐lead ECGs from 43 051 patients, of whom 1413 (3.3%) had > moderate AS. Patients in the derivation data set were all‐comers to a cardiovascular teaching hospital. Those with AS were more often female, with notably low mean body mass index (24 kg/m²) in a Korean population, and often with other structural and functional heart abnormalities such as altered systolic and diastolic function, left ventricular (LV) mass, and pulmonary pressure. While high‐quality (500 Hz) ECG data were used, just 7 basic measures of strictly conventional ECG were employed, along with demographic factors, to build and test the multilayer perceptron as well as statistical pattern recognition models including logistic regression. At the same time, state‐of‐the‐art signal processing was used to generate the convolutional neural network. Standard statistics (area under the receiver operating characteristic curve) were then used to evaluate and internally validate the trained models in the same hospital and in 10 865 patients from a second, community‐based hospital. Results were impressive with an AUC of 0.86 (sensitivity 80% and specificity 78%) in the external validation group. In fact, even a single ECG lead had an AUC of 0.82.

It is worth noting that despite much recent excitement regarding artificial intelligence applications to ECG, there are still very few studies of this kind, especially performed at this scale and across 2 sites with true external validation. Difficulties in securing high‐quality, well‐annotated data, as well as sufficiently sophisticated data curation and handling skills to ensure proper predictions from large sets of often biased data, likely contribute to the paucity of such studies. Nonetheless, Kwon et al. have shown that with a robust DL pipeline, it seems possible to use digital ECG to reasonably accurately predict AS in ways that few if any individual clinicians could.

Utilizing a variety of techniques, several researchers have also recently shown that other cardiac pathologies such as LV systolic dysfunction,5, 6, 7 prior atrial fibrillation,8 LV hypertrophy9 (ideally better redefined electrically as “LV electrical remodeling”),10 and hypertrophic cardiomyopathy11, 12 can also be predicted from 12‐lead ECGs. So with some high‐quality studies of this kind showing promising results, 1 question to be asked is “why have these techniques, especially the DL techniques that have elicited the most excitement, not yet entered clinical practice”? One potential reason is that there are even fewer prospective, randomized studies of DL‐derived tools that demonstrate improved healthcare resource utilization, cost, or patient outcomes.13 Furthermore, while Kwon et al. and others have made attempts to demonstrate specific predictive features of DL algorithms (eg, via sensitivity [saliency] mapping or “attention heatmaps”), such secondary procedures do not rid the DL techniques of their obscure, “black box”‐type features.14 The multiple abstractions and convolutions within the types of DL applied by Kwon et al. and others often defy full transparency or explainability, 2 elements considered essential by some for the ethical use of artificial intelligence.15, 16 So without these elements, the potential for bias, including ethnic bias or the entrenched bias present within any data set used for training, is quite real. For example, it has been shown that ECG features vary by both race and sex.17 Also, although DL models for some conditions such as LV systolic dysfunction have thus far translated fairly well to other ethnicities, the model used by Kwon et al. for AS must be validated not only in other ethnic groups, but also in other clinical groups who will undoubtedly have different types and/or degrees of cardiac comorbidities, as well as different types and levels of ECG background noise, ECG sampling rates, etc.

Moreover, although DL has increasingly “come of age,” it must also be understood that more trusted and fully transparent tools such as logistic regression demonstrate AUC results that are not inferior to, and sometimes superior to, those of the DL techniques,18 something that we suspect might have also occurred in the study of Kwon et al. had more powerful advanced ECG parameters been derived and carefully “feature selected” into their logistic regression procedures.7 Moreover, other statistical pattern recognition techniques such as advanced ECG‐related discriminant analysis can also often add critical discriminatory power for separating 1 cardiac pathology from another, something that until proven otherwise, DL techniques such as that of Kwon et al. might have comparative difficulty accomplishing. With rigorous derivation and careful study of multiple discrete advanced and conventional ECG measures together, it is usually possible to readily identify the key features that serve as the main predictors within one's logistic regression, discriminant analysis, or other forms of ECG‐based statistical pattern recognition. These other techniques can therefore allow one to “discard the black box,” or, at a minimum, to open up one's black box for appropriate scrutiny and transparency. Also, the careful use, even alone, of powerful advanced ECG features now easily derived from any 12‐lead ECG (eg, the spatial QRST angle),19 is also more practical and potentially universally applicable for the arguably most important task of predicting clinical outcomes. Moreover, discrete features of advanced ECG have also recently demonstrated genetic associations in genomewide association studies, with potential pharmacogenomic implications.20

A final issue worth noting before clinical implementation of DL‐type techniques is what the implications of the delivery of distributable systems would be when magnified to scale. Here basic epidemiological statistics and modeling might help. For a condition with prevalence of 3.3%, and a diagnostic tool with sensitivity 80% and specificity 78%, the likelihood of any test being positive is 23%. For any positive result, the likelihood that it is a true positive is only 11%, with the vast majority of positive calls being false. Thus, blindly following such an artificial intelligence result with a referral to echocardiography would rapidly swamp any system's ability to cope. However, careful use within a population with a high prevalence of the disease would be a different story. For example, use in a murmur clinic, or in conjunction with point of care ultrasound or phonocardiography or biomarkers, could also provide a means of further improving diagnostic performance.

In conclusion, the study by Kwon et al. provides a noteworthy advance within the ECG field. At the same time, however, excitement about the potential for clinical use of ECG‐based DL algorithms must tempered by the recognition of their limitations, obscuration, and need for further validation, as well as the existence of equally accurate techniques that are possibly more trustworthy, understandable, and ethical.

Disclosures

Dr Schlegel is a principal of Nicollier‐Schlegel SARL, Trélex, Switzerland, a company that performs clinical and research‐related advanced ECG consultancy. The remaining authors have no disclosures to report.

J Am Heart Assoc. 2020;9:e016193 DOI: 10.1161/JAHA.120.016193.