Abstract
We present the novel use of a deep learning–derived technology trained on the skilled hand movements of cardiac sonographers that guides novice users to acquire high-quality bedside cardiac ultrasound images. We illustrate its use at the point of care through a series of patient encounters in the COVID-19 intensive care unit. (Level of Difficulty: Beginner.)
Key Words: artificial intelligence, cardiac ultrasound, COVID-19, deep learning, point of care ultrasound
Abbreviations and Acronyms: AI, artificial intelligence; COVID-19, coronavirus disease-2019; ECMO, extracorporeal membrane oxygenation; EF, ejection fraction; ICU, intensive care unit; IVC, inferior vena cava; LV, left ventricle; POCUS, point-of-care ultrasound; RV, right ventricle; SARS-CoV-2, severe acute respiratory syndrome-coronavirus-2
Graphical abstract
The global pandemic caused by severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) has forced health care providers to find ways to deliver care to those ill with coronavirus disease-2019 (COVID-19) while minimizing transmission to staff. Artificial intelligence (AI) has simplified tasks that previously required advanced levels of human training, and AI-enabled tools are finding their way into all aspects of clinical medicine. Here, we present the use of a novel deep learning–derived technology, Caption Guidance (Caption Health, Inc., San Francisco, California), which was deployed in the COVID-19 intensive care unit (ICU) by critical care physicians with clinical experience but no formal training in ultrasound to obtain point-of-care ultrasound (POCUS) cardiac images, illustrated by a brief discussion of a series of cases.
Learning Objectives
-
•
To describe the use of novel deep learning–derived technology that guides novice users to acquire high-quality cardiac ultrasound images at the bedside by using prescriptive guidance.
-
•
To describe real-world use of this technology in the COVID-19 ICU and illustrate how it affected decision making in patient care.
Caption Guidance was created by using more than 5,000,000 hand movements by cardiac sonographers to train a convolutional neural network to understand the impact of ultrasound probe movement and position on image appearance. The final model contains more than 7,500,000 parameters and was validated in a 2-center 240-patient pivotal trial (1). It was granted de novo authorization with breakthrough status by the Food and Drug Administration in February 2020 (2). It provides real-time prescriptive guidance to steer the user’s transducer position and hand movements to acquire the desired cardiac ultrasound image while displaying the current image quality and automatically capturing the image when appropriate. It has the ability to automatically detect ejection fraction (EF) independent of chamber volumes with accuracy similar to that of the standard volume dependent approach (3) (Figure 1).
Figure 1.
The Caption Guidance Technology Interface (Caption Health, Inc., San Francisco, California)
Annotated comments in pink highlight the key features of the technology, including the dynamic guidance, the quality meter, and autocapture feature, as well as the automated ejection fraction measurement (AutoEF).
Images from the AI-guided ultrasound machine were uploaded to the picture archiving and communication system for cardiology attending overread. We highlight the integration of the AI into the flow of clinical care, describe key features of the technology, and show how the data influenced decision making. Reviewing these cases may suggest how this technology could be expanded to additional units within the hospital utilizing POCUS, including the emergency department, hospital wards, and perioperative medicine, and potentially limit sonographer exposure to patients with COVID-19. This technology may also have a role outside the walls of the hospital, particularly in resource-limited settings where cardiac sonographers are not readily available.
Case 1
The first case is of a 65-year-old woman with a history of rheumatic heart disease complicated by severe mitral regurgitation with mitral valve replacement with subsequent valve-in-valve transcatheter mitral valve replacement, heart failure with preserved EF, and pulmonary hypertension. She was admitted to the COVID-19 ICU with acute hypoxemic and hypercapnic respiratory failure and clinical concern for COVID-19 despite an initial negative polymerase chain reaction assay. Her POCUS AI-guided cardiac ultrasound scan demonstrated a severely dilated and dysfunctional right ventricle (RV), new from a previous formal echocardiographic study, a hyperdynamic left ventricle (LV), the known mitral valve prosthesis (Figure 2A), and a dilated inferior vena cava (IVC) (Figure 2B). Progressive pulmonary hypertension with subsequent right ventricular failure was determined to be the primary insult as she remained negative for SARS-CoV-2, and she improved with heart failure management. Video 1 shows the apical 4-chamber view, and Video 2 shows the subcostal IVC view.
Figure 2.
A 65-Year-Old Woman With Acute Hypoxemic Respiratory Failure, Secondary to Right Ventricular Failure From Progressive Pulmonary Hypertension
Bedside cardiac point-of-care ultrasound revealed (A) a severely dilated and dysfunctional right ventricle (RV), new from previous imaging, a hyperdynamic left ventricle (LV), a mitral valve prosthesis placed by transcatheter mitral valve replacement (TMVR), and (B) a dilated inferior vena cava (IVC).
Case 2
The second case is of a 76-year-old woman with late-stage small cell lung cancer who was admitted to the COVID-19 ICU with acute hypoxic respiratory failure from suspected COVID-19 pneumonia. Bedside AI-guided cardiac ultrasound generated parasternal long-axis (Figure 3A), apical 4-chamber (Figure 3B), subcostal 4-chamber (Figure 3C), and subcostal IVC views (Figure 3D), revealing normal right ventricular function but new, severe left ventricular dysfunction with a noncollapsible IVC. An automated EF was estimated to be 27% by the deep learning–derived technology. Video 3 shows the parasternal long-axis view, and Video 4 shows the apical 4-chamber view. Her SARS-CoV-2 testing returned negative, and the treating clinicians changed the treatment strategy to a regimen for acute heart failure. Her condition worsened, and she died after a transition to comfort-focused care.
Figure 3.
A 76-Year-Old Woman With Acute Hypoxemic Respiratory Failure From Acute Left Ventricular Failure
Bedside artificial intelligence–guided cardiac ultrasound revealed new severe left ventricular dysfunction (automated ejection fraction estimated to be 27%) and normal right ventricular function with a noncollapsible inferior vena cava. (A) Parasternal long-axis, (B) apical 4-chamber, (C) subcostal 4-chamber, and (D) subcostal inferior vena cava views are provided.
Case 3
The third case is of a 72-year-old man with a history of interstitial lung disease secondary to asbestosis who was on home oxygen therapy. He was admitted to the COVID-19 ICU for acute on chronic respiratory failure as well as shock. Bedside AI-enabled POCUS revealed preserved left ventricular function, a dilated RV with severely reduced function, and a moderate pericardial effusion (Figure 4). The subcostal 4-chamber view is shown in Video 5 and the apical 4-chamber view is shown in Video 6. Subsequent right-sided heart catheterization showed severe pulmonary hypertension with mean pulmonary artery pressure of 47 mm Hg with a moderately reduced cardiac index of 2.0 l/min/m2. Right-sided heart catheterization a month earlier showed a normal cardiac index. His SARS-CoV-2 test result was negative, and he was started on dobutamine for cardiogenic shock. He ultimately developed a hospital-acquired infection and died of progressive shock.
Figure 4.
A 72-Year-Old Man With Acute Hypoxemic Respiratory Failure and Progressive Shock From Acute Right Ventricular Failure
Artificial intelligence–enabled point-of-care ultrasound showed severe right ventricular (RV) dysfunction, normal left ventricular (LV) function, and a moderate circumferential pericardial effusion in the (left) apical 4-chamber and (right) subcostal 4-chamber views.
Case 4
The fourth case is of a 72-year-old man with heart failure with a recovered EF, atrial fibrillation, and mild coronary artery disease. He had fevers, rigors, and cough and was admitted to the COVID-19 ICU with respiratory failure and shock from presumed COVID-19. He received aggressive fluid resuscitation and was initially on high doses of vasopressors and broad-spectrum antibiotics without clinical improvement. A bedside AI-guided cardiac ultrasound scan revealed acute biventricular heart failure with apical and midventricular akinesis, suggestive of severe stress-induced cardiomyopathy (Figure 5). Parasternal long-axis (Video 7) and apical 4-chamber (Video 8) views are provided. His full infectious work-up returned negative results, including SARS-CoV-2 testing. His respiratory failure and shock state improved with diuresis and inotropic support from predominantly noncatecholamine vasopressors.
Figure 5.
A 72-Year-Old Man With Acute Respiratory Failure and Shock From Biventricular Stress Cardiomyopathy
Bedside cardiac ultrasound showed severely reduced biventricular function with contraction only in basal 25% of the left ventricle (LV) and right ventricle (RV) (arrows).
Case 5
The fifth case is of a 52-year-old man with COVID-19–related severe acute respiratory distress syndrome requiring venovenous extracorporeal membrane oxygenation (ECMO). He was transferred to our institution for lung transplant evaluation. Sudden hemodynamic deterioration prompted an AI-guided bedside cardiac ultrasound, which showed a severely dilated and dysfunctional RV with a hyperdynamic LV (Figures 6A to 6C), as well as a dilated, noncollapsible IVC (Figure 6D). A parasternal short-axis view showing flattening of the intraventricular septum is depicted in Video 9, and right ventricular dilation and dysfunction are highlighted in a subcostal 4-chamber view in Video 10. He was urgently converted to venoarterial ECMO and was supported on ECMO for 5 days. He was successfully converted back to venovenous ECMO but was unable to be weaned from it. His condition deteriorated after a massive intraparenchymal hemorrhage, and he eventually died.
Figure 6.
A 52-Year-Old Man With Multisystem Organ Failure and Shock From COVID-19 Who Was Undergoing Venovenous Extracorporeal Membrane Oxygenation Support
Artificial intelligence–guided bedside cardiac ultrasound prompted by sudden hemodynamic collapse showed (A to C) a severely dilated and dysfunctional right ventricle with a hyperdynamic left ventricle, as well as (D) a dilated, non-collapsible inferior vena cava, resulting in an urgent upgrade to venoarterial extracorporeal membrane oxygenation support.
Conclusions
We have highlighted the use of a novel deep learning–derived technology that guides novice ultrasound users to acquire high-quality cardiac ultrasound images at the point of care through prescriptive guidance. This guidance is delivered using insights gained from training a convolutional neural network on millions of observations of the skilled movement of cardiac sonographers while completing thousands of studies. We show how its use at the bedside can affect decision making and patient care. Our discussion showcases the use of AI in medicine, which can simplify tasks that traditionally required highly trained individuals to complete. As our knowledge of AI algorithms increases, we will undoubtedly see more and more of their use in clinical practice.
Funding Support and Author Disclosures
Dr. Narang has previously received honoraria from Caption Health for unrelated work. Dr. Thomas has received consulting and grant support from Edwards, Abbott, GE Medical, and Caption Health; and his spouse is employed by Caption Health. Drs. Cheema and Walter have reported that they have no relationships relevant to the contents of this paper to disclose.
Acknowledgments
The authors would like to acknowledge Caption Health for providing the AI-guided echocardiography machines used in this work and the IDP Foundation, Inc., for their kind contribution that helped build the research facility where this project was completed.
Footnotes
The authors attest they are in compliance with human studies committees and animal welfare regulations of the authors’ institutions and Food and Drug Administration guidelines, including patient consent where appropriate. For more information, visit the Author Center.
Appendix
For supplemental videos, please see the online version of this article.
Appendix
Case 1. Apical 4-Chamber View
Case 1. Subcostal Inferior Vena Cava View
Case 2. Parasternal Long-Axis View
Case 2. Apical 4-Chamber View
Case 3. Subcostal 4-Chamber View
Case 3. Apical 4-Chamber View
Case 4. Parasternal Long-Axis View
Case 4. Apical 4-Chamber View
Case 5. Parasternal Short-Axis View
Case 5. Subcostal 4-Chamber View
References
- 1.Narang A., Bae R., Hong H. Acquisition of diagnostic echocardiographic images by novices using a deep learning based image guidance algorithm. J Am Coll Cardiol. 2020;75:1564. [Google Scholar]
- 2.U.S. Food and Drug Administration 2020. FDA authorizes marketing of first cardiac ultrasound software that uses artificial intelligence to guide user. https://www.fda.gov/news-events/press-announcements/fda-authorizes-marketing-first-cardiac-ultrasound-software-uses-artificial-intelligence-guide-user Available at:
- 3.Asch F.M., Poilvert N., Abraham T. Automated echocardiographic quantification of left ventricular ejection fraction without volume measurements using a machine learning algorithm mimicking a human expert. Circ Cardiovasc Imag. 2019;12 doi: 10.1161/CIRCIMAGING.119.009303. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Case 1. Apical 4-Chamber View
Case 1. Subcostal Inferior Vena Cava View
Case 2. Parasternal Long-Axis View
Case 2. Apical 4-Chamber View
Case 3. Subcostal 4-Chamber View
Case 3. Apical 4-Chamber View
Case 4. Parasternal Long-Axis View
Case 4. Apical 4-Chamber View
Case 5. Parasternal Short-Axis View
Case 5. Subcostal 4-Chamber View