Greasing the Skids: Deep Learning for Fully Automated Quantification of Epicardial Fat

See also the article by Commandeur et al in this issue.

Joe Schoepf, MD, FACR, FAHA, FNASCI, FSCBT-MR, FSCCT, is a professor with appointments in radiology, medicine, and pediatrics at the Medical University of South Carolina (MUSC) in Charleston. At MUSC, Dr Schoepf serves as the director of the division of cardiovascular imaging and vice chair for research, as well as assistant dean for clinical research. His main scientific interest is the use of advanced CT, MRI, image postprocessing, and artificial intelligence techniques for diagnosing disorders of the heart and lung.

Andres Abadia, PhD, MSc, is a postdoctoral fellow in the department of radiology and radiological sciences at the Medical University of South Carolina; here, Dr Abadia’s research focuses on cardiothoracic applications of artificial intelligence and dual-energy CT. Dr Abadia received his MSc degree in biomedical engineering (medical physics concentration) from the University of Florida on May 2012 and concluded his PhD dissertation work on July 2019 to receive his degree in medical sciences (medical physics concentration).

The whole is greater than the sum of its parts.

—Aristotle (384–322 BC)

Now that the initial specter of intelligent machines replacing human medical imagers has been sufficiently litigated and found implausible, we can focus on the things that artificial intelligence (AI) and machine learning (ML)–based algorithms can actually do for us. For example, we know that machines are better than mere-mortal risk prediction schemes to estimate an individual’s likelihood to die (1). Machines are able to make such predictions by creating inferences that may be too complex for the human mind; yet, for their reasoning, they depend on lots of data (2). Some of these data can be simply culled from an individual’s medical record. Computer algorithms currently are being developed that will automate such information harvests.

Particularly rich sources of data are CT scans, and when it comes to risk prediction, nothing beats a CT scan of the heart. However, if one wants to mine the wealth of data contained within a cardiac CT study, that means a yeoman’s work of painstaking manual labor, starting with the calcium score, functional measurements, atherosclerotic plaque characterization, and so forth. That is where the machines come in. That is where AI will show its greatest initial benefit, by delivering us from the drudgery of mundane, menial quantitation tasks. Increasingly, step-by-step, we are teaching our AI minions to extract each morsel of information, each biomarker that can be derived from a heart CT. We taught machines how to estimate blood flow across a coronary artery stenosis (3,4), and we let them take care of the pesky calcium quantification (5).

In this issue of Radiology: Artificial Intelligence, Commandeur and colleagues (6) take aim at the AI-based extraction of yet another biomarker that is inherent in cardiac CT studies, the quantification of epicardial adipose tissue (EAT). EAT has demonstrated relation to early atherosclerotic plaque formation with development of noncalcified plaque and high-risk plaque features and has demonstrated prognostic value for improved risk assessment and outcome prediction (7,8). However, implementing EAT quantification into routine clinical risk stratification pathways has not gained traction to date because of the time-consuming and burdensome quantification steps that current software requires to obtain this biomarker. The current reference standard method of EAT quantification, which has been shown to be reliable between observers, involves the manual segmentation of multiple images within a study, often analyzed and modifiable in the axial view only (9,10).

The conceptual goal of automating EAT quantification is by no means new. We and others have dabbled in this topic over the decades (9,10) using the tools of those times. In the current study, the authors take the topic into the era of AI and report their experience with a deep learning (DL) convolutional neural network (CNN) model for robust and fully automated quantification of EAT. DL, as a subset of ML and a special type of artificial neural network, uses a multilayer approach that leads to higher levels of data abstraction and improved predictions from said data (2). The authors pitted the algorithm against three expert cardiovascular imagers. A total of 850 datasets derived from multiple cohorts, CT systems, and protocols were analyzed. Furthermore, a subset of 141 scans was analyzed for interobserver variability. Additionally, EAT progression was evaluated in 70 patients at baseline and follow-up using both DL and manual quantification by the expert readers. The authors report that EAT automated quantification took an average of 1.57 seconds ± 0.49 (standard deviation) per scan when using graphics processing unit computation and 6.37 seconds ± 1.47 with a central processing unit only, compared with an average of 15 minutes for the manual measurements performed by the expert readers. Importantly, good correlation was shown between the results from this method and the manual results obtained by an expert reader. In the present study there was high agreement of DL with expert manual quantification for all scans (R = 0.97, P <.001) with no significant bias (0.53 cm³, P =.13). Interestingly, EAT quantification using DL revealed high association with progression of noncalcified plaque volume using serial scanning, which was in line with results from manual assessment by expert readers. This highlights the potential of the method as intended and bodes well for a future fully automated approach to EAT quantification by AI-based algorithms.

As experience shows, dedicated imaging applications can be as beneficial as they may be and still see no widespread clinical implementation if they require a modicum of additional unreimbursed labor. The ability to obtain valuable biomarkers such as EAT automatically through AI-based methods drastically enhances the viability of approaches that aim to add value by squeezing all obtainable information from an imaging study. Thus, we are making good progress in chipping away on the task of giving machines all the information they need to dictate patient management. They can tell us whether the pressure gradient across a coronary artery stenosis is great enough for a patient to benefit from revascularization. They can utilize coronary calcium, plaque characteristics, EAT, and a host of biomarkers yet to be discovered, to ever more accurately predict the time of our expiration, or when precisely we will have a heart attack. Yet, currently we are still fumbling in a dark age of fragmentation. A CNN here, an ML algorithm there—but all disparately and erratically aimed at a single isolated feature or problem at a given moment in time.

The next necessary evolutionary step will consist of integrating the sum of all parts into a whole—in the creation of single unified platforms that holistically compute and compose all flow of information on a patient or a population, be it from imaging or otherwise. Doing so will allow painting a complete, whole picture of individuals’ exact state of health at each given time point and allow us to identify the precise needs of each patient, now and in the future.