Will Artificial Intelligence Replace the Human Echocardiographer?

An older population with an increased prevalence of cardiovascular disease and an aging workforce are engendering a state of healthcare crisis in cardiology.¹ Most cardiologists now face an unprecedented time crunch as they rush through their appointments to perform and interpret more and more procedures. The need to multitask creates exhaustion leading to burnout and frequent reporting errors.² The recent interest in using artificial intelligence techniques, such as machine learning, may offer a solution to reduce physician workload including repetitive and tedious tasks involved in diagnosing and analyzing patient data and imaging. To this end, the study by Zhang and colleagues³ in this issue of Circulation adds to the growing enthusiasm for developing a machine learning algorithm that automates several facets of echocardiography measurement and interpretation.

Zhang and colleagues used a deep learning model that has enjoyed spectacular success in addressing computer vision problems including image classification, face recognition, robot navigation, and driverless cars to name a few. Although traditional machine learning workflow includes an initial stage of feature engineering and selection from the data for classification, deep learning methods attempt to learn the important features directly from the raw image data (with minimal preprocessing). Zhang and colleagues applied an algorithm that has triumphed in image recognition tasks and reported a 96% accuracy for distinguishing between broad echocardiographic view classes (eg, parasternal long axis from short axis, or an apical view) and an 84% accuracy overall (including partially obscured views). These results are consistent with a recent study that applied deep learning with convolutional neural networks for view classification of echocardiograms.⁴ However, Zhang et al notably used a deeper architecture with more layers (18 versus 11), considered a larger number of echocardiography view classes (23 versus 15 views), and applied their technique to a larger data set (14 035 versus 267 echocardiographic studies).

The authors also reported an overall metric of accuracy of image segmentation ranging from 72% to 90% for image segmentation. Although deep learning has been explored previously for segmenting the left ventricle,⁵ the work by Zhang et al was much more extensive with additional cardiac segments beyond the left ventricle and a larger data set, involving millions of images from 14 035 studies. Moreover, the authors succeeded in going a step beyond simple classification and segmentation by providing an algorithm for automated quantification of cardiac structure and function. The comparison with manually recorded measurements, however, showed wide limits of agreements emphasizing the potential real-world variability of echocardiography measurements. Independent verification from a core laboratory or the use of a gold standard like cardiac magnetic resonance was not reported and could perhaps help us understand the accuracy and precision of the new technology. Nevertheless, automated view detection, segmentation, and measurements are important advances that have been made increasingly available in recent years.⁶

Finally, the authors assessed the ability to perform disease detection essentially based on the results of the automated quantification. With the output restricted to 2 classes, disease and control, authors were able to identify 3 cardiovascular diseases with impressive results: area under the receiver operating characteristic curve of 0.93 for hypertrophic cardiomyopathy, 0.87 for cardiac amyloidosis, and 0.85 for pulmonary arterial hypertension. These remarkable results expand the observations made in several previous studies where machine learning was touted to have abilities in detecting different cardiac phenotypes.^7–10

Transthoracic echocardiographic imaging is the most commonly performed noninvasive cardiac procedure. Image quality, however, varies substantially between patients and is also operator dependent, which increases interobserver variability. Consequently, despite the wide utilization, errors in echocardiography quantification and interpretation are prevalent and often associated with conflicting interpretations in echocardiography reports.¹¹ In the majority of instances, such errors may be linked to physician fatigue, impaired attention, memory, and executive function that decrease reader recall and attention to detail. There is little doubt that machine learning techniques like the one illustrated by Zhang and colleagues³ would go a long way for better data organization, improving workflows, and reducing echocardiography measurement and cognitive errors. Moreover, these technologies may offer solutions for limited training opportunities and the lack of expert supervision. Undoubtedly, deep learning has the potential of improving clinical workflow and diagnostic efficacy in echocardiography. It is not surprising, as the figure shows, that there has been a recent surge in interest in the application of artificial intelligence techniques in medicine and, in particular, in echocardiography. The figure shows that machine learning and deep learning techniques are rapidly emerging as the preferred approach over traditional methods that use feature engineering. However, the question that is looming currently is whether deep learning techniques could eventually supersede an echocardiographer? This question pleads for an understanding of the current state of deep learning and artificial intelligence technologies in relation to human intelligence. The strengths and limitations of the current deep learning techniques have been recently debated.¹² First, the bio-inspired neural network design used currently for deep learning remotely resembles the activities of a human brain. Structurally, they resemble at best the outer layers of the retina or the visual cortex where images are just sensed or represented. These layers of neurons can hardly do what our brains are capable of, such as the reasoning and knowledge components represented in the more open terrains of general intelligence, that form a foundation of pervasive common-sense knowledge in humans. There are several other innate virtues of human cognition. Human learning is distinguished by its richness and its efficiency; a child learns rapidly to ambulate after a few falls. Moreover, humans can use a learned model to orchestrate action sequences that maximize future reward. However, a deep learning algorithm would require thousand-to-millions of attempts at learning before it can accurately relegate an image! Thus deep learning, although widely publicized for its amazing performance, is actually quite shallow in its intelligence. Ironically enough, the deep in deep learning refers to the architecture of the neural network and not the concept of learning as has been well highlighted in a recent treatise by Gary Marcus.¹²

Figure. Trends in artificial intelligence techniques in echocardiography based on publications in PubMed.

echo indicates “echocardiography” or “echocardiogram” or “cardiac ultrasound”; FE, “feature extraction” or “feature engineering”; ML, “machine learning”; DL, “deep learning”; “+,” logical AND. PubMed publication counts obtained using the EUtilsSummary() function in the R package “RISmed” (https://CRAN.R-project.org/package=RISmed).

From the foregoing account, it is clear that complex decisions like determining the appropriateness of a test or the mechanism to weigh and extract information from study in the clinical context would be quite arduous for current computational algorithms. A physician uses his tacit and codified knowledge of cardiac physiology to target measurements in end-systole and end-diastole and integrates these measurements in a mental model. A deep learning technique may not be programmed to extrapolate the knowledge of cardiac physiology, but rather depend on mundane features that may not make intuitive sense beyond the data sets. It is astounding how expeditiously physicians learn incipient pathologies and disease presentations. We develop a hierarchical pattern of new knowledge representation based on association with previous knowledge. This allows us to quickly identify the uniqueness of a new case. Deep learning techniques currently cannot relegate such cognizance hierarchically; most correlations between sets of features are flat or nonhierarchical, as if in a simple, unstructured list, with every feature often treated with similar priority.¹² Thus, unlike human intelligence, deep learning cannot yet integrate prior knowledge effectively (at least, in the same way as a human), and in the absence of large data sets, it would be difficult to train the network to correctly identify rare anomalies. That said, it is worth considering that the most appropriate approach used by machine leaning techniques to reach their decisions may not always follow the same steps that are often used by humans. Clearly, this is an area where cross talk between new cognitive, unsupervised, deep learning, and reinforcement learning approaches may be especially paramount for future development.

In summary, the work presented by Zhang and colleagues represents a ladder in the right direction; however, it is still a long journey ahead. In the current time, the best use of the technology would be to free up time for physicians from repetitive low-level and uneventful activities like measurements, data preparation, standardization, and quality control to more direct time spent in higher calibers of interpretation, patient care, and medical decision making. This will perhaps allow physicians to be more interactive and experiential in answering diagnostic questions, and communicating the findings optimally to teams, a key ingredient of good patient care. Moreover, such strategies can embolden the patient-doctor relationships and overcome any engineered approaches that carry the risks of dehumanizing care. A well-functioning patient-physician encounter is an essential part of healing, in particular, for chronic disorders where the skills of physicians and diagnostic tests can influence patients’ objective and subjective measures of well-being.^13,14 Perhaps by efficaciously integrating the automated measurements and interpretations, the human physician will continue to do what machines are incapable of (or at least, not yet good at), bring higher wisdom in personalized care, foster patient engagement, and motivate patients for much needed chronic lifestyle behavior changes.