Artificial intelligence and imaging: Opportunities in cardio-oncology

Abstract

Cardiovascular disease is a leading cause of death in cancer survivors. It is critical to apply new predictive and early diagnostic methods in this population, as this can potentially inform cardiovascular treatment and surveillance decision-making. We discuss the application of artificial intelligence (AI) technologies to cardiovascular imaging in cardio-oncology, with a particular emphasis on prevention and targeted treatment of a variety of cardiovascular conditions in cancer patients. Recently, the use of AI-augmented cardiac imaging in cardio-oncology is gaining traction. A large proportion of cardio-oncology patients are screened and followed using left ventricular ejection fraction (LVEF) and global longitudinal strain (GLS), currently obtained using echocardiography. This use will continue to increase with new cardiotoxic cancer treatments. AI is being tested to increase precision, throughput, and accuracy of LVEF and GLS, guide point-of-care image acquisition, and integrate imaging and clinical data to optimize the prediction and detection of cardiac dysfunction. The application of AI to cardiovascular magnetic resonance imaging (CMR), computed tomography (CT; especially coronary artery calcium or CAC scans), single proton emission computed tomography (SPECT) and positron emission tomography (PET) imaging acquisition is also in early stages of analysis for prediction and assessment of cardiac tumors and cardiovascular adverse events in patients treated for childhood or adult cancer. The opportunities for application of AI in cardio-oncology imaging are promising, and if availed, will improve clinical practice and benefit patient care.

1. Introduction

Among the ~17 million cancer survivors in the United States, cardiovascular disease is a leading cause of death [1], [2], [3], [4], [5]. Cancer patients and survivors are at risk of cardiovascular toxicity, and many already have underlying cardiovascular issues that can complicate cancer therapy and impair outcomes. Prediction and early recognition of cardiovascular diseases in this population are crucial and can potentially inform treatment and surveillance decision-making. Furthermore, as survival and the number of potentially cardiotoxic cancer therapies expand, so does the demand for high-throughput, yet high-quality, focused imaging modalities to serially monitor cardiovascular function and structure in these patients [2], [6], [7]. Additionally, cancer patients are often excluded from clinical trials and in general tend to be treated in a more conservative manner versus non-cancer patients [8]. Novel, more efficient methods of obtaining these data, such as with the use of AI, are desirable to aid in the assessment of cardiac function in the cardio-oncology patient population. By extracting hidden patterns and evidence from large amounts of healthcare data, artificial intelligence has the potential to generate novel predictors and indices in cardio-oncology patients. The field of AI-assisted precision cardio-oncology is therefore evolving toward greater personalization and precision, with a strong emphasis on early prevention and tailored treatment prior to, during, and after cancer treatment.

Artificial intelligence (AI) refers to computer programs that are capable of performing tasks associated with human intelligence, such as pattern recognition and problem-solving. AI has emerged as a rapidly advancing field that is beginning to have an impact on clinical practice, particularly regarding identification of established data patterns, which can then be used to predict new outcomes. Within the field of cardiovascular medicine, researchers are discovering that AI has a potential role in cardiac imaging, with applications ranging from image classification and reconstruction to segmentation and quantification automation, all of which have the potential to impact workflow, diagnostic accuracy, measurement reproducibility, and ultimately patient prognosis.

Here we focus on such AI technologies applied to cardiovascular imaging in cardio-oncology with a strong emphasis on prevention and tailored treatment of a variety of cardiovascular conditions for cancer patients (Table 1). We discuss AI in echocardiography as the most common form of imaging in cardio-oncology, and much work is also being done applying AI to echocardiography in cardiology. We then propose the use of AI-augmented cardiovascular magnetic resonance (CMR), computed tomography (CT) (with a focus on coronary artery calcium (CAC) scans), single positron emission computed tomography (SPECT), and positron emission tomography (PET) imaging. We also suggest AI-augmented multimodality imaging to assess intracardiac tumors. This article is one of a series of publications on AI in cardio-oncology, complementing our manuscript on the application of AI to electrocardiography and biologically relevant models in precision cardio-oncology for the prediction and modeling of cardiovascular adverse events in cancer survivors. In our companion manuscript, titled “Artificial Intelligence Opportunities in Cardio-Oncology: Overview with Spotlight on Electrocardiography,” we delve deeper into this relatively new field of cardio-oncology especially describing therapies of special interest and the types of cancers for which they are used [9].

Table 1.

Potential utility of artificial intelligence in imaging in cardio-oncology.

Imaging in cardio-oncology	Utility before cancer treatment	Utility during cancer treatment	Utility after cancer treatment
Echocardiography	Establishment of baseline cardiac assessment using automated LVEF and GLS measurements. Predicting CV outcomes with ML algorithms to guide decision-making. AI-guided echo acquisition can expand the use of echo to primary care and oncology settings	Follow-up cardiac assessment using automated LVEF and GLS measurements to predict CV outcomes with ML algorithms to guide decision-making; AI-guided echo acquisition can expand the use of echo to primary care, oncology, and other settings	Follow-up cardiac assessment using automated LVEF and GLS measurements to predict CV outcomes using ML algorithms to guide decision-making; AI-guided echo acquisition can expand the use of echo to primary care, oncology, and other settings
	AI can facilitate the detection of subtle abnormalities in TTE that may not be visually seen by an interpreting cardiologist to improve prognostic/diagnostic accuracy	AI can facilitate the detection of subtle changes in TTE that may not be visually seen by an interpreting cardiologist to improve prognostic/diagnostic accuracy	AI can facilitate the detection of subtle changes in TTE that may not be visually seen by an interpreting cardiologist to improve prognostic/diagnostic accuracy
Cardiovascular magnetic resonance imaging	AI approaches applied to CMR can facilitate efficient diagnostic performance for cardiac amyloidosis, simulating CMR reading by experienced operators	AI approaches applied to CMR can facilitate efficient diagnostic performance for cardiac amyloidosis, simulating CMR reading by experienced operators	AI approaches applied to CMR can facilitate efficient diagnostic performance for cardiac amyloidosis, simulating CMR reading by experienced operators
	Successful application of AI to CMR tissue characterization using radiomics and texture analysis can improve prognostic and diagnostic accuracy of subtle abnormalities in the myocardium	Application of AI to CMR to address etiological concerns can be key to identifying cardiovascular toxicity and can be crucial to inform decisions to cease or continue cancer therapy or initiate immunosuppression	Successful application of AI to CMR tissue characterization using radiomics and texture analysis can improve diagnostic accuracy of imaging scar, wall thickening differentiation, and inflammation
Computed tomography (CAC)	Chest CT obtained for planning cancer treatments can be automated to assess CAC which is a robust target for primary cardiovascular risk reduction	Chest CT previously obtained for planning cancer treatments can be automated to assess CAC which is a robust target for primary cardiovascular risk reduction	Chest CT previously obtained for planning cancer treatments can be automated to assess CAC which is a robust target for primary cardiovascular risk reduction
	Cancer surveillance chest CT can be automated to assess CAC which is a robust target for cardiovascular risk reduction	Cancer surveillance chest CT can be automated to assess CAC which is a robust target for cardiovascular risk reduction	Cancer surveillance chest CT can be automated to assess CAC which is a robust target for cardiovascular risk reduction
Single proton emission computed tomographya	ML algorithms can be applied to SPECT to provide additional neutrality (supplementing subjective assessments by reading clinicians) in processing data relating to myocardial perfusion	ML algorithms can be applied to SPECT to provide additional neutrality (supplementing subjective assessments by reading clinicians) in processing data relating to incident myocardial perfusion	ML algorithms can be applied to SPECT to provide additional neutrality (supplementing subjective assessments by reading clinicians) in processing data relating to evolving or incident myocardial perfusion
	Combining ML algorithms with SPECT can improve prediction accuracy in the determination of baseline cardiac abnormalities for high-risk patients	Combining ML algorithms with SPECT can improve prediction accuracy in the determination of short-term adverse cardiac effects especially for high-risk patients	Combining ML algorithms with SPECT can improve prediction accuracy in the determination of long-term adverse cardiac effects especially for high-risk patients
Positron emission tomographya	MACE and myocardial ischemia can be challenging to predict and might gain from ML to clarify baseline risk assessment	MACE and myocardial ischemia can be challenging to predict and might gain from ML to clarify evolving cardiac injury and ongoing prognosis	Using ML algorithms in conjunction with cardiac PET can augment the detection of damage to coronary arteries post-radiation
	–	AI can automate PET scan assessment of new inflammation resulting from cancer immunotherapy	AI can automate PET scan assessment of persistent inflammation resulting from cancer immunotherapy
Multimodality imaging	Automation of detection and characterization, including analysis of size, shape, and textural patterns, of tumors can define and refine the diagnosis through incorporation of data from CT, MRI, FDG-PET and large image databases	Monitoring response to treatment by tracking size and texture of tumors, and presence of any additional tumors, can be automated, with incorporation of data from CT, MRI, FDG-PET and large image databases	Post-treatment monitoring can be automated for surveillance of size, texture, and presence of recurrent or additional tumors, through incorporation of data from CT, MRI, FDG-PET and large image databases
	Incorporation of AI algorithms can help determine prognosis and treatment of masses in or near the heart	Incorporation of AI algorithms can help optimize prognosis and treatment of masses in or near the heart	Incorporation of AI algorithms can help optimize prognosis and treatment of masses in or near the heart

AI = artificial intelligence; CAC = coronary artery calcification; CMR = cardiac magnetic resonance; CT = computed tomography; CV = cardiovascular; CVD = cardiovascular disease; FDG-PET = Fluorodeoxyglucose (FDG)-positron emission tomography; GLS = global longitudinal strain; LVEF = left ventricular ejection fraction; MACE = major adverse cardiovascular events; ML = machine learning; MRI = magnetic resonance imaging; PET = positron emission tomography; SPECT = single-photon emission computerized tomography; TTE = transthoracic echocardiography.

^aUse of SPECT and PET in cardio-oncology is currently limited and may expand in the future.

1.1. Cardiovascular imaging

The rapid growth of advanced multimodality cardiovascular imaging has generated massive amounts of data that have transformed cardiovascular care. The steps that slow down the process include timing and the accuracy with which these images are interpreted [10]. Artificial intelligence applications in cardiovascular imaging have demonstrated enormous promise in terms of diagnostic support and image interpretation [11]. Acquiring high-quality imaging to feed into AI algorithms for image interpretation presents a unique set of challenges. This requires image registration and segmentation. Registration is used to align multiple images, correct artifacts, rotate the image, and ensure that all images have the same orientation in order to create a consistent and complete source of information [34]. Segmentation is the process of extracting content from images by identifying landmarks, segmenting them into meaningful segments, and identifying regions of interest. In the literature, advanced AI-driven segmentation techniques have been described for a variety of imaging modalities and clinical applications [12], [13]. After registering and segmenting the appropriate structures, automated measurements can be taken [14], [15], [16]. Numerous large national and international multicenter imaging databases have been established, and images have been pre-registered and segmented, making them suitable for machine learning applications [17], [18]. Other smaller studies have been conducted manually at individual centers or using retrospective imaging data from observational studies or randomized trials [19], [20].

2. Echocardiography

Transthoracic echocardiography (TTE) is an essential tool in cardio-oncology to assess ventricular, atrial, valvular, and pericardial structure and function in patients with current or past cancer. Currently, the echocardiographic assessment of left ventricular ejection fraction (LVEF) and global longitudinal strain (GLS) plays a central role in the diagnosis and monitoring of cardiotoxicity from cancer therapy [21]. However, there are limitations of the current workflow, such as length of analysis, inter-operator and inter-observer variability that can result in man-made variation and limited reproducibility. Furthermore, conventional assessment of LVEF and GLS can only reflect cardiotoxicity in the myocardium based on traditional definitions and may miss more subtle signs of dysfunction.

Numerous studies have thus evaluated the interpretation of echocardiograms using AI [22], [23], [24]. Within cardio-oncology, artificial intelligence algorithms may play an important role in echocardiography, with uses ranging from image classification and reconstruction, automation in segmentation and quantification, to risk prediction with integration of demographic and medical data, all of which can potentially impact efficiency, accuracy in diagnosis, reproducibility in measurements, and ultimately patient prognosis (Fig. 1, Table 2). AI methods such as machine learning could improve the efficiency of obtaining cardiac function measurements and early diagnoses without compromising reliability.

Fig. 1.

Opportunities for the application of artificial intelligence to echocardiography in Cardio-Oncology include automation of left ventricular function assessment and strain, as well as real-time AI-guided image acquisition particularly with point-of-care tools at the bedside, in the examination room, or in low resource settings.

Table 2.

Machine learning artificial intelligence techniques applied to imaging modalities in cardio-oncology.

Imaging in cardio-oncology	Artificial intelligence techniques	Reference
Echocardiography	Machine learning (ML)-enabled software (AutoLV, TomTec-Arena 1.2, TomTec Imaging Systems, Unterschleissheim, Germany)	[25]
	Deep learning (DL), convolutional neural network (CNN), image segmentation	[26]
	DL, CNN, artificial intelligence (AI)-guided image acquisition software (Caption Guidance)	[28]
	CNN, DL model (EchoNet)	[22]
	Ensemble ML model with three different ML algorithms (support vector machine (SVM), random forest (RF), and artificial neural network (ANN))	[29]
	Supervised ML algorithm (least absolute shrinkage and selection operator (LASSO) methods with bootstrap resampling)	[27]
Cardiovascular magnetic resonance	Fast strain-encoded CMR imaging (fast-SENC) using MyoStrain analysis software, feature tracking (FT)	[31]
	SVM with Gaussian radial basis function (RBF) kernel (RBF-SVM), texture analysis, segmentation	[35]
	DL-based algorithm within the Circle Cardiovascular Imaging Inc. software, segmentation	[36]
	DenseNet-121 (CNN), FT	[34]
	AI – Workstation EWS Cardiac Analysis Software, Philips Achieva 3.0 T TX	[43]
	Video-based echocardiography model, 2D-CNN based model, 3D-CNN based model	[44]
	DL, ML	[45]
	AI enhanced electrocardiogram, deep neural network (DNN)	[46]
	AI-based myocardial texture analysis, SVM	[47]
Cardiac computed tomography	Computer-aided detection (CAD), CADstream, Merge, Hartland, WI, USA	[49]
	Supervised ML: k-nearest neighbor (kNN), linear classifier (LC), SVM, RF, boosting, ANN DL, CNN	[50]
	DL algorithm, CNN	[51]
	AI-based, automatic coronary artery calcium (CAC) scoring software	[52]
	End-to-end DNN, three-dimensional (3D) CNN model, Tri2D-Net	[53]
	DL algorithm	[54]
	Four CAD systems: CAD 1 (Lung VCAR version 11.3–10.11; GE Healthcare, Milwaukee, Wis): automatic segmentation CAD 2 (ImageChecker CT version 8.3.12; R2 Technologies, Sunnyvale, Calif) CAD 3 (Syngovia Via Va 20; Siemens Medical Solutions, Forchheim, Germany): Anatomical Intelligence CAD 4 (Cornell Via; Cornell University, Ithaca, NY)	[62]
Nuclear cardiac imaging	SVM, ML DL, CNN	[48]
	ML, ensemble boosting with LogitBoost (using decision stumps and RF)	[56]
	ML, boosted ensemble algorithm, LogitBoost, Waikato Environment for Knowledge Analysis (WEKA) platform	[17]
	DL, deep CNN	[59]
Multimodality imaging of masses	ML, supervised ML, RF, SVM, regression, logistic regression, DL, unsupervised DL, CNN, deep CNN, automated segmentation algorithm, AI-based monitoring, Computer Aided Nodule Assessment and Risk Yield (CANARY), texture analysis	[61]
	CAD, computer-aided diagnosis	[63]
	Unsupervised clustering	[64]
	Unsupervised DL, deep belief network (DBN)	[65]
	Supervised feature selection algorithm	[66]
	Automatic segmentation, brain tumor image analysis (BraTumIA)	[67]
	Supervised ML: ANN, SVM, decision tree, RF, Naive Bayes classifier, fuzzy logic, and kNN Unsupervised ML: clustering and association rule-learning algorithms Reinforcement machine learning DL: recurrent neural network (RNN), CNN, and DNN Cognitive computing	[68]

AI = artificial intelligence; ANN = artificial neural network; CAD = computer-aided detection; CMR = cardiovascular magnetic resonance, CNN = convolutional neural network; DL = deep learning; DNN = deep neural network; FT = feature tracking; kNN = k-nearest neighbor; ML = machine learning; RF = random forest; SVM = support vector machine.

Machine learning uses computer algorithms that are capable of learning and adapting without explicit instructions, by analyzing and inferring patterns in data using advanced statistical models to determine output. For example, automated LVEF measurements utilizing AI-assisted point-of-care echocardiography at the time of oncology or infusion center appointments have the potential to significantly streamline care for patients who require serial LVEF assessments. In the FAST-EFs, a multicenter study of 255 patients, automated left ventricular (LV) measurements were feasible, rapid, and reproducible compared to visual and manual Simpson's biplane method. The average analysis time for automatic LV measurements was 8 ± 1 s/patient, and there was no inter- or intra-observer variability [25]. Automated strain measurements have been less well studied. In a study of 152 patients with human epidermal growth factor receptor-2 (HER2)-positive breast cancer treated with anti-HER2 therapy and anthracyclines, automated ejection fraction and GLS were obtained via AI assistance; these measurements were in close agreement (median standard deviation of strain values 1.2%) with standard software-derived values on serial echocardiographic monitoring [26]. Some vendors have also developed point-of-care tools that integrate strain through fully automated or offline strain measurements.

In current clinical practice, echocardiograms are obtained by trained sonographers and overread by echocardiographers. Recent work has evaluated nurses without training in echocardiography or sonography guided by AI-algorithms to obtain echocardiograms [28]. If the guiding and measurement algorithms proved to be robust, the ability to obtain LVEF and GLS at the bedside before chemotherapy infusions by oncology nurses with some cardiology oversight may be the future of a fully integrated and collaborative approach to cardio-oncology care. An essential component of the success of this workflow, however, is the necessity to maintain engagement between cardiologists and the oncology team to best care for the cardio-oncology patients. Other AI guided echocardiographic findings such as detecting intracardiac masses, pericardial effusions, and utilizing inferior vena cava imaging for right atrial pressure estimation may prove helpful in the cardio-oncology population and are topics of ongoing investigations.

Abundant imaging markers embedded in echocardiograms or other medical data can be identified by AI to define new functional indices and improve diagnostic and prognostic accuracy. This information may be currently overlooked or not fully utilized due to the limitation of computational power and our understanding of imaging markers. A deep learning model (EchoNet) trained on a data set of more than 2.6 million echocardiogram images from 2850 patients was used to identify local cardiac structures, estimate cardiac function, and predict systemic risk factors such as age and weight [22]. Machine learning algorithms incorporating speckle-tracking echocardiographic data have also been applied for automated discrimination of pathological remodeling in hypertrophic cardiomyopathy from physiological hypertrophy seen in athletes [29]. This suggests that AI can be utilized to identify and distinguish echocardiographic changes in patients with cancer therapy-related cardiac dysfunction compared to healthy hearts that may not be seen by visual inspection of the interpreting cardiologist. This theory was investigated using machine learning algorithms to discover patterns of strain features most strongly associated with cardiotoxicity in a longitudinal prospective cohort study of 248 breast cancer patients receiving doxorubicin chemotherapy. Machine learning algorithms were able to identify cardiac mechanics abnormalities related to a decline in LVEF in this population [27]. Finally, there is hope in the future capability of AI (using machine learning and natural language processing, techniques used to mine clinical documentation) to integrate all medical data including imaging to predict prognosis.

Finally, applying AI to picture archiving and communication systems [30] allows for further cost-reduction and improvements in process efficiency via personalized workstation image arrangement, automated electronic medical record data entry, and report preparation. Advances in AI image interpretation have now made it possible for automated re-analysis of stored Picture Archiving and Communication System (PACS) images, which may translate into more accurate reporting and a reduction in inter- and intra-observer variability [30].

3. Cardiac magnetic resonance imaging

Cardiovascular magnetic resonance imaging offers gold standard assessment of ejection fraction and non-invasive tissue characterization which can yield some of the most practically important information to address treatment decisions in particular with regard to the use of ongoing potentially cardiotoxic cancer therapy [31]. Studies have suggested that left ventricular function and global longitudinal strain are better assessed using CMR than 2D echocardiography. CMR also gives better views of the right ventricle, which can be injured during cancer therapy [32], [33]. In addition, cardiovascular magnetic resonance imaging can yield important information to address etiological concerns [31]. For patients with cancer who receive a diagnosis of cardiovascular disease during or after cancer therapy, the importance of teasing out the underlying etiology of the cardiovascular diagnosis is of vital significance and crucial to decision-making regarding cessation or continuance of cancer therapy.

Multiple roles for AI in CMR are currently being explored [34], [35], [36] (Table 2). Fully automated cardiac localization and image plane planning/acquisition is now a commercial reality and can substantially decrease scan and analysis time while also correctly identifying image artifacts, applying fixes, or triggering repeat image acquisitions. AI applications in parallel and real-time imaging and compressed sensing have allowed for more rapid image acquisition without compromising diagnostic accuracy. The successful application of AI to CMR tissue characterization using radiomics and texture analysis has improved diagnostic accuracy of scar imaging, wall thickening differentiation, and inflammation [35].

It is certain that many applications of AI to CMR will apply to cardio-oncology. CMR is increasingly utilized for the evaluation of cardiotoxicity and cardiac pathology in oncology patients. Deep learning algorithms have been applied to CMR to enable accurate and fully automated analysis of LV volumes and function [36]. Feature tracking, tagging and fast-strain-encoded CMR techniques are emerging means to assess myocardial strain using CMR [34]. CMR is also preferred for detailed tissue characterization and/or scar detection, the non-invasive interrogation of cardiac masses, and perfusion imaging. Machine learning has been applied to improve efficiency in magnetic resonance fingerprinting, an emerging tool that allows for quantification of several tissue specific parameters such as T1, T2, and T2* relaxation times in a simultaneous, unified and streamlined, single multi-parametric scan [37].

4. Cardiac computed tomography

Cardiac CT provides a platform for promising AI application, including CAC scoring on ECG-gated non-contrast chest CT (Table 2). CAC has a well-defined role in screening patients for coronary artery disease [49] and for assessing the risk of major adverse cardiovascular events (MACE). Dedicated CAC scoring utilizes an ECG-gated chest CT exam which may incur extra cost and resource. Fortunately, CAC scoring can be reliably assessed from non-gated chest CT scans and has high reproducibility and excellent concordance with ECG-gated cardiac CT. Detection of CAC on non-contrast CT scans used in cancer surveillance may be used in cardiovascular risk assessment and potentially improve adherence and uptake of cardiovascular prevention strategies. Currently, there is a reliance on staff trained to perform CAC scoring and interpret CAC qualitatively by a categorical method (none, mild, moderate, or severe coronary artery disease). Artificial intelligence is a promising tool for not only for opportunistic detection of atherosclerotic disease in this population, but also implementation of more qualitative methods which may be more accurate and reproducible.

Multiple methods of automated CAC scoring using have been validated as highly accurate [50]. AI- based detection (using convolutional neural networks) and measurement of CAC scoring was studied in one group of breast cancer patients [51]. A standard CAC scoring algorithm was applied to the data originally used to train the algorithm. Each patient was assigned to one of the five standard CAC risk categories (0, 1–10, 11–100, 101–400, and >400). The performance of the automated calcium scoring was evaluated against manual CAC score measurement. Automated CAC scoring using AI showed high reproducibility (linearly weighted kappa 0.85; 95% CI: 0.77–0.93), high agreement for CAC score categories against the test set (proportional agreement of 0.87; 95% CI: 0.79–0.92) and an even higher intraclass correlation coefficient (ICC) for continuous CAC (ICC 0.95; 95% CI: 0.93–0.97). In another observational study of 315 consecutive, non-contrast CT scans, AI-based semi-automatic and automatic software were obtained for three CAC scores (Agatston score, volume score, mass score) and number of calcified lesions which had excellent correlation and agreement [52].

Using AI, a highly reliable and actionable cardiovascular disease risk profile can be achieved in subjects undergoing treatment planning or follow-up of cancer from their existing non-contrast chest CT. A deep learning cardiovascular risk prediction model trained on 30,286 low dose CT scans from the National Lung Cancer Trial was able to identify patients with high cardiovascular mortality (AUC of 0.768), thereby converting the low dose CT scan for lung cancer screening to a tool for cardiovascular risk assessment [53]. The implementation of AI CAC tools on low dose CT scans for lung cancer, will potentially allow for more accurate evaluation of CAC and determination of cardiovascular risk, a comprehensive preventative approach [54]. Another known risk for premature coronary artery disease in oncology patients is exposure to high dose chest radiation. Coronary artery dose-volume parameters have been evaluated to predict risk of calcification in patients who have received radiation therapy [55]. Larger studies addressing the accuracy of AI-based CAC and atherosclerotic disease assessment from planning chest CT in breast cancer patients are also forthcoming. These results could be extrapolated to patients with other malignancies who undergo non-gated CT chest for treatment planning or surveillance.

5. Nuclear cardiac imaging

AI has recently been used to assess prognostic markers in nuclear cardiology (Table 1). Risk prediction using machine learning applied to PET scans was more effective at identifying patients at high risk of myocardial ischemia and/or MACE than logistic regression, using the SCORE risk model based on European Society for Cardiology guidelines [56]. As patients who have received certain cancer treatments (i.e. chest radiotherapy) are at a higher risk of MACE and cardiac ischemia, this combined technique could be useful for monitoring ischemic heart disease in cancer patients. Additionally, cardiac PET in another study was used to illustrate that coronary flow reserve (CFR) inversely correlates with radiation dose to particular coronary regions such as the left anterior descending artery (R = −0.5, p = 0.002) [57]. These results suggest that cardiac PET may identify damage to coronary arteries following radiation therapy. Application of AI to cardiac PET scans to evaluated myocardial perfusion in cardio-oncology patients is therefore an emerging avenue.

Immune checkpoint inhibitors (ICIs) are monoclonal antibodies to a variety of immune checkpoint regulators. ICIs induce cytotoxic T-cells that were previously dormant to recognize and target cancer cells. Fluorodeoxyglucose F 18 (¹⁸F-FDG)-PET scans of twenty patients treated with programmed cell death protein 1 (PD-1) inhibitors, cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) inhibitors or combination therapy were analyzed before and after therapy with ICIs [58]. These patients showed marked increases in ¹⁸F-FDG PET uptake in the ascending, descending, and abdominal aortas, aortic arch, and iliac arteries, suggesting increased inflammatory activity in large arteries likely secondary to activated local T cells, which can contribute to destabilization of atherosclerotic plaques [58] and contribute to MACE. AI could be used to track changes in the distribution of tagged ¹⁸F-FDG over time. If these changes could be characterized and correlated with chemotherapy treatments and cardiovascular outcomes, preventative measures could be taken to reduce these changes in future patients.

Machine learning adds objectivity to the reading of myocardial perfusion SPECT imaging [17] (Table 2). In 2619 consecutive patients referred for exercise or pharmacological stress testing, physician diagnosis was compared with machine learning predictions and automated perfusion quantification indexes (stress and ischemic total perfusion deficit) [17]. Visual analysis of SPECT by physicians was scaled between 0 and 4, while coronary artery disease likelihood was also reported 0–2 (low to high) [17]. Automated perfusion quantification indices were generated by traditional imaging software to correspond with the shape of the myocardium [17]. Ejection fraction, systolic and diastolic volumes at stress and rest were assessed by the software [17]. The studied population had a 9.1% 3-year MACE rate with a total annual MACE rate of 3% [17]. Prediction of MACE using machine learning combined with both clinical and imaging data variables was superior to the existing visual or automated perfusion quantification assessments [17]. The enhanced predictive valve and objective assessment powered by AI are particularly important for longitudinal monitoring of cancer patients undergoing cardiotoxic therapies.

The ability of deep learning to predict obstructive cardiac disease from myocardial perfusion imaging (MPI) against prediction by computer-calculated total perfusion deficit (TPD) alone was assessed in 1638 patients with known coronary artery disease who underwent stress testing and coronary artery angiography [59]. The obstructive disease was defined as greater than or equal to 70% blockage in any artery or left main artery stenosis greater than or equal to 50%. Overall, deep learning was more sensitive in predicting obstructive coronary artery disease than TPD alone. With deep learning set to the same specificity as TPD, sensitivity using deep learning was higher at 82.3%, compared to 79.8% for TPD without deep learning. Per vessel, sensitivity increased 5.4% using deep learning instead of TPD. TPD is considered as an equivalent standardized surrogate for expert reading in the detection of coronary artery disease [59].

6. Special cases for multimodality imaging

6.1. Cardiac masses

Cardiac primary tumors are extremely rare (0.001–0.3% in autopsy series, most commonly myxomas), however, the prevalence of metastasis is much higher, and has been detected in up to 9.1% of patients with known malignancies [60]. Of importance, in cardio-oncology in particular, is the differentiation of cardiac tumors and thrombi. Artificial intelligence has not been applied to cardiac masses but a significant body of literature exists in the application of AI to other tumors (see review [61]) (Table 2).

Artificial intelligence can optimize the use of cardiac imaging for masses at multiple levels, including detection, characterization, and monitoring. The imaging assessment of cardiac masses includes the analysis of the size, shape and textural patterns of the tumor. Artificial intelligence is especially robust in recognizing complex patterns in an image, including some not detected by the human eye. Based on deep learning in particular, leading to differentiation of healthy and cancerous tissue, AI is able to precisely measure the size and shape of the mass, and delineate its margins [49]. Artificial intelligence is able to identify regions with suspicious patterns on CT alone or with additional information from FDG-PET and present them to the readers [62]. AI can furthermore help to characterize the mass: by incorporating the knowledge of large image databases and including clinical, genetic, pathology data, AI can refine the diagnosis of the mass [63], [64], [65], [66]. Additionally, algorithms can be incorporated, helping to determine the prognosis of the mass and to optimize its treatment. Once the diagnosis has been made, imaging patterns such as the variability of the imaging signal, reflecting the heterogeneity of the mass may play a role in the prognosis [66]. Finally, AI can help monitor the response to treatment, tracking the size, the texture, and the presence of additional tumors [67].

6.2. Cardiac amyloidosis

Cardiac amyloidosis is caused by the buildup of misfolded proteins in the myocardium, resulting in restrictive cardiomyopathy that can lead to heart failure, conduction system dysfunction, and cardiac mortality [44]. The major subtypes of cardiac amyloidosis are transthyretin (ATTR) amyloidosis resulting from misfolded transthyretin protein and light chain (AL) amyloidosis resulting from deposition of misfolded immunoglobulin light chains [44]. Early detection is paramount for cardiac amyloidosis in order to initiate treatment prior to advanced progression of disease.

Machine learning and deep learning approaches have been applied to CMR and have shown great diagnostic performance (AUC 0.982) for diagnosing cardiac amyloidosis, simulating cardiovascular magnetic resonance reading by experienced operators [45]. Research is also ongoing for other imaging modalities. Artificial intelligence-enhanced electrocardiography can enhance early detection of cardiac amyloidosis [46]. AI-based myocardial texture analysis using echocardiography has aided in diagnostic specificity [47]. Models combining electrocardiographic data with echocardiographic data have also demonstrated promising results [44]. Additionally, the potential role for artificial intelligence to improve image analysis, disease diagnosis, and risk prediction in cardiac amyloidosis is also emerging for nuclear cardiology [48].

6.3. Myocarditis

Immune checkpoint inhibitors (ICIs) are novel therapeutics used to treat cancer by activating immune cells, particularly T lymphocytes, to more readily target cancer cells. At the same time, ICIs can also arm the immune system against healthy tissues [38]. The activity of some ICIs display cross-reactivity with cardiac proteins such as titin, which leads to inflammation of heart tissue (myocarditis) [39]. Myocardial changes associated with ICI treatment are often initially subclinical without overt symptoms, leading to difficulty with making a clear diagnosis. Additionally, it can be difficult to determine whether early cardiotoxic changes are a result of pre-existing cardiac damage [40]. Elucidation of an accurate diagnosis is important in determining whether to continue use of an ICI in cancer treatment. Further, myocarditis during or after treatment with ICIs can be treated effectively with immunosuppression to prevent further cardiac damage if an accurate diagnosis is made [40].

Of note, CMR is most specific for tracking myocardial changes during or after myocarditis [41], [42]. In some studies, AI has been used to identify early changes suggestive of subclinical myocarditis. In one study, early gadolinium enhancement (EGE) was evaluated in addition to left ventricular functional parameters using artificial intelligence algorithms applied to CMR images from patients with acute myocarditis [43]. In these patients, EGE irregularities were found involving 41 regions in different sites on the myocardium. Findings suggested that the application of artificial intelligence algorithms to EGE on CMR could play an important role in screening patients suspected to have acute myocarditis. This could potentially be extrapolated to the cardio-oncology population - using AI to automate the analysis of gadolinium enhancement on cardiac MRI to detect early changes associated with subclinical myocarditis that may have otherwise gone undetected. Prompt recognition of acute myocarditis, whether clinical or subclinical, associated with chemotherapy can be critical for early life-saving cardioprotective treatment.

7. Discussion

Utilization of AI in cardiovascular imaging can streamline and optimize the workflow for staff, providers, and the healthcare system, and increase the diagnostic power and accuracy of the images. AI is being used in multiple fields to streamline complex analyses from datasets to glean useful trends and information. Given proper datasets, AI can be a useful tool for clinicians to streamline the imaging process. Scans guided by AI to help personalize patient care would prove especially useful in the field of precision cardio-oncology, as cancer therapy-induced cardiotoxicity could be assessed in a timely manner with more careful monitoring over time (Fig. 2). In the future, an approach incorporating genetic, clinical, and imaging data implemented by machine learning or AI may aid in understanding the mechanistic underpinnings of adverse cardiotoxic effects and predict the prognosis of the patients.

Fig. 2.

Implementing artificial intelligence in imaging in cardio-oncology clinical practice.

There are some caveats to the field of AI worth noting. AI algorithms use large training datasets, as smaller datasets are prone to error especially when bias is present in the data. Therefore, validation among other datasets may be needed, requiring collaborations among institutions and electronic health records. Further, deep learning analyses with neural networks require capable and efficient supercomputing machines, both costly and time-consuming. Deep learning with multiple layers may also increase the algorithm training time for data acquisition without substantial improvement in precision [68].

AI is poised to revolutionize cardio-oncology preventive care. Artificial intelligence algorithms are educated on current data to forecast complicated outcomes learned by the algorithm. Artificial intelligence has been applied in healthcare for a variety of purposes, including tumor diagnosis and staging. However, we must be aware of the potential that AI models will reflect, perpetuate, or even promote bias in medicine. To ensure that AI models are generalizable to a wide range of people and that their implementation does not reflect or perpetuate healthcare disparities, great caution must be taken. The potential for algorithms to limit resource allocation and attention to racial and ethnic minority patients in comparison to Caucasians is a major source of worry, since it reflects trends in the data used to train the algorithms [69], [70]. This is particularly crucial in cardio-oncology, given the increased incidence of cardiovascular adverse effects from cancer therapies in racial and ethnic minorities, particularly African Americans, compared to Caucasians [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81]. This concern can be potentially overcome by properly retraining the algorithms [70] and also addressing underlying sources of bias. This, however, raises concerns about the long-term applicability and impact of algorithms that are not carefully monitored and iterated to account for evidence of bias and the impact of social determinants of health. A method based on distributive justice may be able to ease these fears of spreading bias and unfairness [69]. With a distributive justice approach, prediction models would be trained on datasets that are more inclusive of minority populations, ensuring the potential for equitable patient outcomes (where these minority groups benefit from the model in the same way as their counterparts), equal performance (ensuring model accuracy across non-homogenous groups), and equal resource allocation (to correct racial disparities) [69]. Consequently, during this revolutionary period of innovation in cardio-oncology, it is critical that we advocate for inclusion to ensure that gaps in health outcomes are improved rather than worsened [82], [83]. This concept applies to bias regarding racial and ethnic minorities, as well as regarding women. Algorithms must be trained on data substantially including women if they are to be applied to women.

Although most current AI models have achieved high accuracy in internal validation, external validation using independent cohorts is critical before implementation in patient care. Due to the lack of benchmarking of datasets and the complexity of regulatory science, there is much to do to implement AI models for cardio-oncology. Visible or explainable machine learning approaches may offer potential solutions to enhance the characterization of cardio-oncology patient heterogeneity compared to traditional “black-box” AI models. National and international efforts to standardize clinical and imaging data are also needed in the near future to optimize data for use in AI algorithms in cardio-oncology.

CRediT authorship contribution statement

Conception and design: SAB, NM, MSC.

Drafting of the manuscript: SAB, NM, JL, NA, PC, FC, AG, LZ, AS, IN, EW, NG, MSC.

Interpretation of data: SAB, NM, JL, NA, PC, FC, AG, LZ, AS, IN, EW, NG, MSC, AH.

Critical revision: SAB, NM, JL, NA, PC, FC, AG, LZ, AS, IN, EW, NG, MSC, AH.

Final approval of manuscript: All authors.

Funding

This publication was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant Numbers UL1TR001436 and KL2TR001438. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.