Cardiovascular disease/stroke risk stratification in deep learning framework: a review

Abstract

The global mortality rate is known to be the highest due to cardiovascular disease (CVD). Thus, preventive, and early CVD risk identification in a non-invasive manner is vital as healthcare cost is increasing day by day. Conventional methods for risk prediction of CVD lack robustness due to the non-linear relationship between risk factors and cardiovascular events in multi-ethnic cohorts. Few recently proposed machine learning-based risk stratification reviews without deep learning (DL) integration. The proposed study focuses on CVD risk stratification by the use of techniques mainly solo deep learning (SDL) and hybrid deep learning (HDL). Using a PRISMA model, 286 DL-based CVD studies were selected and analyzed. The databases included were Science Direct, IEEE Xplore, PubMed, and Google Scholar. This review is focused on different SDL and HDL architectures, their characteristics, applications, scientific and clinical validation, along with plaque tissue characterization for CVD/stroke risk stratification. Since signal processing methods are also crucial, the study further briefly presented Electrocardiogram (ECG)-based solutions. Finally, the study presented the risk due to bias in AI systems. The risk of bias tools used were (I) ranking method (RBS), (II) region-based map (RBM), (III) radial bias area (RBA), (IV) prediction model risk of bias assessment tool (PROBAST), and (V) risk of bias in non-randomized studies-of interventions (ROBINS-I). The surrogate carotid ultrasound image was mostly used in the UNet-based DL framework for arterial wall segmentation. Ground truth (GT) selection is vital for reducing the risk of bias (RoB) for CVD risk stratification. It was observed that the convolutional neural network (CNN) algorithms were widely used since the feature extraction process was automated. The ensemble-based DL techniques for risk stratification in CVD are likely to supersede the SDL and HDL paradigms. Due to the reliability, high accuracy, and faster execution on dedicated hardware, these DL methods for CVD risk assessment are powerful and promising. The risk of bias in DL methods can be best reduced by considering multicentre data collection and clinical evaluation.

Introduction

The global mortality due to cardiovascular diseases (CVDs) is 18 million yearly, leading to a financial burden of $237 billion USD per year (1-6). With the COVID crisis still not over, an increase in depression, a rise in comorbidities, environmental changes, the Ukraine-Russian war, and its effects on people, all the above have accumulated further stress, thereby an increase in CVD (7). While CVD plaque formation starts with low-density lipoprotein (LDL) penetration in the walls via the epithelial layer of the arterial walls due to endothelial dysfunction, however, the formation of atherosclerosis over time in the walls of the coronary arteries blocks the blood flow leading to cardiovascular events (8). The comorbidities of different kinds such as diabetes (9), chronic kidney disease (10,11), rheumatoid arthritis (RA) (12,13), hypertension (14), hyperlipidaemia (15-17), and brain diseases (18,19), causes an increase in plaque burden, putting the individual in a vulnerable state of heart disease and stroke, leading to an increase in mortality. Therefore, there is a need for early detection of CVD to reduce mortality rates.

The CVD risk prediction is being carried out by using popular conventional calculators tools such as QRISK3 (20), Framingham risk score (21), the systematic coronary risk evaluation score (SCORE) (22), Reynolds risk score (RRS) (23), and the atherosclerosis cardiovascular disease (ASCVD) (24). While the calculators follow specific guidelines given by the American College of Cardiology/American Heart Association (25), the European Society of Cardiology (ESC) (26,27), and the Canadian Society (28,29). There are several challenges, such as (I) over simplification of the relationship between the risk factors and gold standard, (II) not well-defined granularity in risk classes for better statin control, (III) no direct quantitative solution for plaque type and plaque burden/thickness measurements in the walls of the arteries, (IV) lack of heterogeneity (multiple ethnicities) in the populations, and (V) lastly, does not leverage the knowledge of cohort (30-35). All the above reasons make the CVD risk prediction results less reliable and accurate.

Due to the similar genetic make of carotid and coronary artery disease, non-invasive imaging of carotid arteries is preferred. The typical imaging paradigms for carotids are computed tomography (CT) (36,37), magnetic resonance imaging (38), and ultrasound (US) (39). cBUS referring to carotid B-mode ultrasound provides us with advantages as such being cost-effective, user-friendly, can easily access the carotid arteries with by the window through the neck, and further, compound, harmonic imaging provides high resolution images (40).

Since CINE loops of the carotid ultrasound can be used with cardiac gating, one can therefore use this for estimating the vulnerability of the plaque build-up (41). The whole process is done by using the image registration paradigms among the slices (42) and then undergoing plaque characterization. The carotid ultrasound image-based phenotypes (CUSIP) which includes carotid intima-media thickness (cIMT) (43-47), intima-media thickness variability (IMTV), maximum plaque height (MPH), and total plaque area (TPA), all these can be measured in cBUS frozen scans. These CUSIP biomarkers (also called a radiomic features) can be fused with office-based biomarkers (OBBM) and laboratory-based biomarkers (LBBM) for superior CVD risk stratification, as demonstrated by AtheroEdge 2.0 (Roseville, CA, USA) (48). Although it is fully automated system, scale-space-based paradigm only took partial advantage of the intelligence such as the (I) width and (II) the edge of the arterial wall by modelling the (i) standard deviation and (ii) derivative of the Gaussian functions (49,50). Such statistical models did not leverage on the knowledge of cohort and framework-based in artificial intelligence (AI). Thus, for a reliable and accurate CVD risk prediction, a superior solution is needed.

In earlier research, machine learning (ML) has been tried for predicting the CVD risk by taking the covariates or risk factors, namely OBBM, CUSIP, LBBM, and MedUse (51-61). Because ML had demonstrated a better ability to handle the non-linearity between the covariates and the gold standard, and ability to integrate advance feature selection methods, such paradigms are therefore superior to previous statistical methods (51-61). Previously, an ML-based CVD risk stratification review was published (62,63), however, did not incorporate the recent advances in deep learning (DL). DL further improves the accuracy of the results by automating the feature extraction process. The principle of the working of the DL techniques is shown by several industries such as Google, and Facebook (64-77). The DL can be categorized into a few types such as solo DL (SDL), hybrid DL (HDL), ensemble DL (EDL), transfer learning (TL), and encoder-decoder (ED)-based DL. This review presents the different DL approaches for prediction of risk in CVD, along with the risk of bias (RoB) among the DL systems.

The layout of the proposed review is as follows. The study selection was conducted using PRISMA model in Section “Search strategy and statistical distribution”. The same section will present the AI-attributes with statistical distribution. Linkage between the atherosclerosis and CVD risk, and a time course study for CVD risk stratification is presented in the section “Biological link between atherosclerosis and cardiovascular diseases and a time course study for cardiovascular risk stratification”. Section “Deep learning architectures for CVD risk stratification” represents the DL architecture for the CVD risk prediction with their performance evaluation metrics. Section “Cardiovascular risk assessment via carotid plaque characterization: brief overview” presents the CVD risk assessment via plaque characterization using carotid US techniques. The CVD risk prediction in today’s COVID-19 framework is also presented in section “Cardiovascular disease risk assessment in COVID-19 framework”. The DL-based CVD systems using Electrocardiogram (ECG) are shown in section “Cardiovascular risk stratification using deep learning-based ECG”. Section “Artificial intelligence explainability in cardiovascular disease” presents the explainable artificial in cardiovascular disease. Section “Risk of bias estimation in deep learning-based cardiovascular disease systems” presents the bias in DL systems for CVD risk estimation. The critical discussions on the review are shown in section “Critical discussion”, and finally the conclusions are presented in section “Conclusions”.

Search strategy and statistical distribution

We aimed at four different types of statistical distributions such as the types of CVD techniques, the ground truths (or gold standard) used for the AI-based systems, the feature extraction strategies adapted, and bias in the AI-based systems. We adopted PRISMA model for selection of the studies used for CVD risk prediction. Further, this section has subparts as section “PRISMA model” and section “Statistical distribution” with the selection criteria and the statistical distribution respectively.

PRISMA model

The search and selection of the final studies were done using the PRISMA model. The search was done with the keywords “CVD risk using deep learning”, “cardiovascular disease and deep learning”, “CVD risk classification in deep learning framework”, “CVD risk prediction using artificial intelligence”, “CVD risk prediction using AI”, “CVD/stroke risk assessment in Deep Learning framework”, “CVD/Stroke using the non-invasive framework”, “CVD/Stroke using hand-held devices”, “Bias in DL/AI for CVD risk stratification”, “signal processing-based Deep Learning methods for cardiovascular disease risk stratification”, “EEG-based Deep Learning methods for CVD risk”, “CVD risk estimation in COVID patients”, and “AI-based techniques for CVD risk stratification in COVID-19 framework”. The different platforms used for searching were Science Direct, IEEE Xplore, PubMed, and Google Scholar. The selection of the studies is shown by the PRISMA flowchart in Figure 1. Using our search strategy, we obtained a total of 2,672 studies. We used three set criteria for our exclusion process: (I) rejection of records due to insufficient data, (II) rejection of studies which were non-relevant, and (III) removal of the articles after screening of the studies. After applying the above-mentioned exclusion criteria, results found were 2,284, 88, and 14 studies for each type of exclusion shown as E1, E2, and E3 (Figure 1). The more relevant and valuable scientific information were obtained from the final selected studies and distribution were created based on statistics. An analysis was done stating the architecture of SDL and HDL techniques, their characteristics, and bias estimation. Further DL paradigm was discussed in ECG, and COVID-19 paradigm.

Figure 1.

PRISMA model for selection of the DL-based CVD studies. DL, deep learning; CVD, cardiovascular disease.

Statistical distribution

The statistical distribution which were obtained from the final studies are presented in Figure 2. The statistical distribution was carried out using the following attributes (a) publications trend over the year, (b) types of DL framework used for CVD risk prediction, (c) the different architectural algorithm followed in the DL-based paradigm, and (d) types of performance evaluation metrics adopted in the CVD risk prediction. The DL-based publications presented an increasing trend over years from 2015–2022 as shown in Figure 2A. The SDL techniques considered more with 69% (13,78-97) when compared to the HDL techniques with 31% (98-108) (Figure 2B). The algorithms used in the DL-based framework consisted of the following distribution, such as convolutional neural network (CNN) (45%), UNet (16%), artificial neural network (ANN) (7%), UNet++ (7%), UNet+++ (3%), DeepMAD (4%), context fusion network (CFNet) (3%), fourier CNN (FCNN) (3%), SegNet-UNet (3%), deep ensemble (3%), deepMedic (3%), feedforward neural network (FNN) (3%), and deep neural network (DNN) (3%). As shown in Figure 2C, the CNN algorithm was followed in most of the studies. During our bias study, we will examine how these architectures are related to the bias in DL solutions for CVD risk stratification. The prediction results are evaluated through the performance evaluation (PE) metrics shown in Figure 2D, where the total number of studies is depicted below for each metric. The different PE metrics were sensitivity (18 studies), specificity (17 studies), accuracy (20 studies), false-positive rate (FPR) (1 study), positive predictive value (PPV) (2 studies), negative predictive value (NPV) (1 study), precision (7 studies), F1-score (1 study), Jaccard index (7 studies), P value (9 studies), humming loss (2 studies), and DICE coefficient (15 studies).

Figure 2.

Statistical distribution (A) publications per year, (B) types of DL paradigm, (C) types of DL algorithms for CVD risk stratifications, (D) types of performance evaluation metrics. DL, deep learning; CVD, cardiovascular disease; HDL, hybrid deep learning; SDL, solo deep learning; ANN, artificial neural network; DNN, deep neural network; FNN, feedforward neural network; FNCNN, fourier CNN; CNN, convolutional neural network; CFNet, context fusion network; DICE, coefficient; FPR, false-positive rate; ACC, accuracy; Sen, sensitivity; Spec, specificity; NPV, negative predictive value; PPV positive predictive value.

Biological link between atherosclerosis and cardiovascular diseases and a time course study for cardiovascular risk stratification

This section describes the biological framework of atherosclerosis and its link with the cardiovascular disease presented in section “Linkage between atherosclerosis and cardiovascular disease”. Then a time course study is discussed in section “A time course study for cardiovascular risk stratification” that involves the description of all five generations of development in the cardiovascular risk stratification field.

Linkage between atherosclerosis and cardiovascular disease

Atherosclerosis disease formation is the fundamental cause leading to CVD (109). Figure 3, left (a) (119) shows the formation of plaque in the walls of the arteries known by the process of atherogenesis. When there is a low endothelial shear stress (8), the process of atherogenesis accelerates.

Figure 3.

Left: atherosclerosis progression consisting of (a) endothelium dysfunction and lesion initiation, (b) formation of a fatty streak, (c) formation of fibrous plaque underlying fibrous cap, and (d) rupture and thrombosis. Right: ultrasound scanning of the carotid artery (both images courtesy of AtheroPoint™, Roseville, CA, USA) (Open Access) (9,11,13,80,110-118). oxLDL, oxidized low-density lipoprotein; SMC, smooth muscle cell.

Figure 3, right shows the artery being scanned using the ultrasound probe. There are several factors which affect the shear stress such as characteristics of the flow velocity which includes direction, flow type, and velocity. Leukocytes in the blood attack the epithelium in this region [Figure 3, left (a)] (8). The monocytes migrate into the sub-epithelial layer, and the oxidation occurs here. The LDL cholesterol then converts into macrophages [Figure 3, left (b)] (120,121). Using the oxidation process of LDL, the macrophages grows into large foam cells, and finally leading to necrotic core formation [Figure 3, left (c)]. The calcium granules expand in the necrotic cells forming calcium deposit lumps. The necrotic are separated from the blood vessels by the fibrous caps (122).The flow of blood is fine till the plaque is small due to the remodelling done by the arteries themselves (123). As the plaques size increases, the volume of the lipid-core reduces resulting to structural stabilization of plaque [Figure 3, left (a)] (124). The fibrous cap gets thin eventually with the deposition of lipids, resulting the rupture of the plaque (125). After the cup rupture the platelets moves into the bloodstream, and the blood clot or thrombus formation occurs turning the arteries to block and atrial stiffness (126). As a result the blood supply is restricted to the tissues causing the cell to die. In the same way, if the blockage occurs in the coronary artery leading to myocardial infarction (MI) or CVD [Figure 3, left (d)] (9,13).

A time course study for cardiovascular risk stratification

There are basically five generations of CVD risk stratifications, listed as follows: (I) Manual calculations, (II) Conventional calculators, (III) image phenotype-based (AtheroEdge™ 1.0, and AtheroEdge™ 2.0); (IV) ML-based system (AtheroEdge 3.0_ML), and (V) DL-based systems (AtheroEdge 3.0_DL).

First generation used manual methods, i.e., did not adopt the computer-based strategy. These CVD risk calculations were made by the physician by directly observing the blood test reports, the family history of the patients, environmental stress on the patients, etc. (Table 1). Further, the manual methods consisted of evaluating the ultrasound exams of the common carotid artery and carotid bulb and understanding the echogenicity, such as echolucent and echogenic grayscale intensities. Since it was a manual method, there was primarily CVD risk stratified into two classes (low and high) and it was not categorized into a multiclass framework (127-129).

Table 1. Generation characteristics for cardiovascular risk stratifications.

SN	Attributes	1st Gen (CVD Manual)	2nd Gen (CVD CC)	3rd Gen (CVD AE)	4th Gen (CVD ML)	5th Gen (CVD DL)
1	Manual	√	×	×	×	×
2	OBBM	×	√	√	√	√
3	LBBM	×	√	√	√	√
4	CUSIP	×	×	√	√	√
5	MedUSE	×	×	√	√	√
6	MRI	×	×	√	√	√
7	CT	×	×	√	√	√
8	US	×	×	√	√	√
9	Multiclass	√	√	√	√	√
10	Multi-label	×	×	×	√	√
11	Ensemble	×	×	×	√	√
12	References	(127-129)	(21,49,50,130-136)	(35,137-140)	(14,48,54,141-145)	(78-80,90,98,99,103)

SN, serial number; Gen, generation; CVD, cardiovascular disease; CC, conventional calculators; AE, AtheroEdge; ML, machine learning; DL, deep learning; OBBM, office-based biomarkers; LBBM, laboratory-based biomarkers; CUSIP, carotid ultrasound image phenotypes; MRI, magnetic resonance imaging; CT, computed tomography; US, ultrasound.

In the second generation, the conventional method for CVD risk stratification, various conventional tools such as FRS, SCORE, and ASCVD were used to determine the CVD risk (21,49,50,130-136). Our group has previously published 11 conventional calculators for CVD risk stratification (146). The covariates used in the second generation were OBBM and LBBM. Different measurement classes were collected based on these risk factors. These measurements had variability with different data sets and were not reliable. The threshold was obtained from these measured values which were ad-hoc in nature. These thresholds could characterize the CVD risk into two classes (low and high).

The third generation consisted of primarily image-based strategies for CVD risk prediction. This used carotid ultrasound image-based phenotypes (CUSIP)-based measurements. Several groups demonstrated the use of cIMT and total plaque area (TPA) for CVD risk stratification. AtheroEdge™ 1.0 was designed for CVD risk based on several risk classes. AtheroEdge 2.0 system was then designed which used the combination of OBBM, LBBM, and CUSIP for 10-year risk prediction and CVD risk stratification (35,137-139). AtheroEdge™ systems used scale-space image processing methods for automated lumen-intima (LI) and media-adventitia (MA) border detection for the far wall of the carotid arteries and several applications were published (140) (Table 1).

The fourth generation consisted of ML-based CVD risk stratifications using the covariates and gold standard. These images can be collected by using MRI, US, and CT (141) modalities. CUSIP was collected using automated advanced segmentation algorithms based on scale-space as in generation three. These CUSIP risk factors were put along with the covariates OBBM, and LBBM risk factors to make it composite, which were then used for training the model. Different types of ML classifiers are used, namely, support vector machine (SVM), random forest (RF), decision tree (DT), and Naïve Bayes (NB), etc. If the gold standard were in multiple classes, then the CVD risk prediction outcome was in a multiclass framework (14,48,54,142-145) (Table 1).

The fifth generation involves DL techniques for automated carotid artery far wall segmentation and then using either generation three or generation four for CVD risk stratification. For the fifth generation, the prediction was done in multiclass, multilabel, and ensemble paradigms (78-80,90,98,99,103). Note that, generation 2^nd and above can be used to monitor response to treatment.

Deep learning architectures for CVD risk stratification

The DL-based systems in the CVD field are in infancy nowadays. With the fast advances in DL for segmentation and classification systems in general, we have started to see its penetration in the CVD area. We have therefore approached the CVD risk stratification systems into four parts: subsection “Point model-based CVD risk stratification using deep learning” details the point model-based CVD risk stratification using deep learning, subsection “Image-based CVD risk stratification using deep learning” describes the image-based CVD risk stratification using deep learning, subsection “Architecture for 2D carotid wall segmentation using solo deep learning” presents the architecture for 2D carotid wall segmentation using SDL, and subsection “Architecture for 2D carotid wall segmentation using hybrid deep learning” talks architecture for 2D carotid wall segmentation using HDL, and subsection “Types of deep learning systems and its characteristics” details the types of the DL systems along with the characteristics in an elaborate way.

Point model-based CVD risk stratification using deep learning

Risk stratification for CVD using machine and deep learning can be accomplished using the combination of point (tabular) and image data models (147). The architecture used for the point data has basically four major components. The first component is dedicated to data preparation and lies along with component two which consists of data partition. The third component is for offline training model generation. The fourth and final component consists of CVD risk prediction (Figure 4).

Figure 4.

CVD architecture using AE3.0DL enveloping multiclass ML/DL-based. SMOTE, synthetic minority over-sampling technique; PCA, principal component analysis; OBBM, office-based biomarkers; LBBM, laboratory-based biomarkers; CUSIP, carotid ultrasound image phenotype; ANN, artificial neural networks; KNN, K-nearest neighbour; SVM, support vector machine; ML, machine learning; GRU, gated recurrent unit; RNN, recurrent neural network; LSTM, long short-term memory; DL, deep learning; ACC, accuracy; BiGRU, bidirectional GRU; CVD, cardiovascular disease; CAD, coronary artery diseases.

The processes involving in data preparation are namely, (I) data normalization by standard scalar paradigm, by which the risk factors (features) get maps between 0 and 1, (II) dominant feature selection through principal component analysis (PCA)-based pooling strategy, and (III) data augmentation by SMOTE model. It is a statistical technique for feature selection which is well preferred in the ML industry. The component two consists of cross-validation system where the system runs K10 protocol, which consists of 90% training and 10% testing. The third component is for offline coefficients generation by DL-based classifiers such as CNN (Figure S1, Appendix 1) (148), LSTM (Figure S2, Appendix 1) (149), RNN (Figure S3, Appendix 1) (150,151) and generative adversarial networks (GAN) (Appendix 1) (152), risk factors and coronary artery syndrome (CAS) serving as input and ground truth respectively to the classifiers. Forth component is the prediction system where the CVD risk is predicted by transforming the test data sets using the training models. The process is carried out in the cyclic order to ensure all the combinations are mutually exclusive and no test data is present in the training pool. The optimization of the training model is also present as an embedded feature in the CVD risk assessment system. The online system contains the performance component for accuracy computation, which uses predicted CVD risk along with the CVD gold standard labels. It consists of receiver operating characteristics and area under the curve (AUC) estimation, along with computation of p-value, highlight the significance of risk factors.

Image-based CVD risk stratification using deep learning

The general online structure of DL architecture basically consists of a data acquisition block, online deep learning model weights, a DNN block for processing online scans, plaque quantification, and a CVD risk stratification system (Figure 5). For the given patient, the carotid ultrasound images were acquired in a real-time, and the CUSIP risk factors were computed from these US scans. This includes plaque burden measurements, far wall IMT (ave., max., min.), IMTV, TPA, and lumen diameter (LD) (153-160). The offline DL-weighted models also act as an input to the online processing unit segmenting the plaque. This is then quantified and overlays generated in colour which is later CVD risk stratified: low CVD risk (green) or high CVD risk (red) risk.

Figure 5.

General DL architecture for CVD risk stratification (Courtesy AtheroPoint^TM, Roseville, USA). DL, deep learning; CVD, cardiovascular disease.

Architecture for 2D carotid wall segmentation using solo deep learning

The basic type of DL architecture is SDL, where only one type of DL architecture gets employed in the risk stratification system. Figure 6 describes the UNet architecture, which was first proposed by Ronnebeger et al. (67). It generally has a U-shaped structure which consists of four encoders and four decoders corresponding to the two sides of the U-shape. The input given to the UNet architectures are red-green-blue (RGB) images, grayscale images, and the masked binary ground truth in the form of a “mini-batch” with a fixed size during the training phase i.e., the number of images processed at one time. The computational ability of the system or hardware determines the mini-batch size. The higher the number of the “mini-batch size”, the higher should be the computational requirement of the hardware computer. The size of the mini-batch mostly consists of numbers that are in the power of 2 i.e., 2, 4, 8, 10, etc., which means these particular numbers of images were processed at one time using the UNet architecture. The encoder and the decoder block in the UNet architecture are described below.

Figure 6.

Solo deep learning (UNet) architecture. ReLU, rectified linear unit.

The encoder block

The encoder blocks are the layers at beginning of the UNet architecture whose purpose it extracts the features of the image. The process of feature extraction is conducted by the process of “convolution” and “ReLU activation function” operations. The last stage of each encoder is the “max pooling” block which chooses the highest features in each filter region, which further down samples the image. Thus, the process of convolution (green), ReLU (light green), and max pooling (yellow) are repeated for each of the blocks of the UNet on the encoder side (see Figure 6). The encoder helps in extracting the features of the grayscale region. The number of filters shown in Figure 6 is 64, which gets double with each passing stage resulting in the count of 128, 256, and 512 filters. Figure 6 shows the numbers 3×3×64, 3×3×128 … 3×3×1024; where 3×3 is the filter size and 64…1024 is the number of filters.

The decoder block

Figure 6 has the decoder stages on the right hand side. It is the reverse of the encoder block. It is needed to recover the original dimensions as used in the training image. Counter to that, the number of filters is reduced to half in each stage of the decoder module, in the way 512, 256, 128, and 64. The decreasing number of filters on the decoder side helps in resizing the image to its original size. After the last decoder stage, the image gains its original size with refined features. Each decoder stage consists of a stack of up-convolution-2D layers (blue), depth-concatenation (turquoise), 2D-convolution (dark green), and ReLU (light green). On the decoder side is the softmax layer which converts the output to a binary image consisting of foreground (white) and background (black) image.

There have been several modifications to the basic UNet by altering UNet components such as the encoder, decoder, skip connection, bridge network, and loss function. These are advanced UNet architectures for 2D and 3D vascular segmentation, which is valuable for CVD risk stratification (98,99,161-163).

Architecture for 2D carotid wall segmentation using hybrid deep learning

The other type of architecture followed in the DL paradigm is HDL. The concept of HDL was introduced by Szegedy et al. (64) where the two types of different architectures were fused together. The interception between the two types of DL systems was done with different AI models. There are a few basic HDL models such as SegNet-UNet, VGG-UNet, ResNet-UNet, and VGG-SegNet. The model used here is SegNet-UNet, which has two SDL models such as SegNet and UNet. The block diagram was presented in Figure 7. The two SDL systems were arranged in a parallel way, which shares a common input, softmax classification, and optimization layers (ADAM). SegNet part of the hybrid model also carries an encoder-decoder-based architecture; however, the stack of layers is different from the UNet model (64). Each encoder stage in the SegNet part consists of a stack of 2D-convolution, batch normaliza­tion, ReLU, and max pooling layers. Also, each decoder stage in same consists of max un-pooling, batch normalization, 2D-convolution­al, ReLU, and max pooling layers. The quantification of the wall plaque burden that was segmented using HDL was also used for risk assessment using AtheroEdge 1.0 (33) or using AtheroEdge 2.0 (48). The wall plaque is one of the attributes that can be used for composite CVD risk computation, which can be further divided into “no-risk”, “low-risk”, “low-moderate”, “high-moderate”, “low-of-high”, and “high-of-high” risk classes for CVD risk stratification. There have been several modifications to the basic hybrid UNet by cascading or parallelizing UNet architecture. These are advanced UNet architectures for 2D and 3D vascular segmentation, which is valuable for CVD risk stratification (86,163-165).

Figure 7.

Hybrid deep learning (SegNet-UNet) architecture. ReLU, rectified linear unit; CE, cross-entropy.

Types of deep learning systems and its characteristics

Deep learning systems for CVD risk prediction are generally get divided into two types (I) SDL and (II) HDL. Our observations showed that SDL has been more adopted compared to HDL systems. The general features of DL systems are presented in Table 2 in chronological order. The general characteristics of the DL systems have been described by using 46 attributes which are divided into 4 clusters namely, demographics (Table 3), DL architecture (Table 4), performance parameters (Table 5), scientific and clinical evaluation (Table 6).

Table 2. General types of DL systems.

SN	Author (year)	SDL/HDL	Basic description
1	Saba et al. (94) (2015)	SDL	The developed an accurate system used CT modality for segmenting and quantifying the calcium regions. The cohort consisted of 75 subjects (one slice per subject). The algorithm used mean shift algorithm along with adaptive threshold mechanism. The performance used four metrics namely correlation coefficient (CC) between manual and automated method (CC=0.978 and PoM of 0.915), dice similarity (mean of 0.85 with SD of 0.085), Jaccard Index (mean of 0.747 with SD of 0.12), and polyline distance metric (mean of 0.195 with S of 0.177)
2	Lakadir et al. (82) (2017)	SDL	CNN algorithms were used for the segmentation of the plaque area. The unique capabilities of DL for segmentation were discussed such as automatic feature extraction. The plaque characterization was implemented using 90,000 patches which were extracted from an image database. The results showed a correlation of 0.90 with clinical assessment suggesting DL approaches are superior
3	Biswas et al. (79) (2018)	SDL	The review concludes that for the CVD/stroke risk assessment, the DL paradigms are superior to non-AI (conventional) paradigms. cIMT measurement was carried by using a two-stage DL system. Stage-I was for feature extraction by a convolution layer-based encoder. Stage-II consisted of ML-based regression that smoothed the LI and MA borders as final output. The DB consisted of 396 B-mode US images. The system demonstrated a 20% improvement in the cIMT readings when compared to sonographer’s readings
4	Sudha et al. (93) (2018)	SDL	The study adopted a two stage process where the stage-I consisted of a ROI extraction using deep learning CNN model, while stage-II consisted of border estimation of LI and MA using a deformable model (so-called snakes). The CNN model used a patch-based model as input for ROI extraction. The low data set consisted of 20 subjects giving a mean error difference of 0.08 mm between manual and automated process
5	Azzorpardi et al. (78) (2019)	SDL	A novel Deep Neural Network-a fully automated tool for segmentation was developed, that deals with both LI and MA boundaries. A geometrically constrained function was proposed. The DICE measured for MA and LI regions was 0.962 and 0.925, respectively
6	Wu et al. (84) (2019)	SDL	For diagnosing the CVD risk, the authors segmented the inner and outer walls of the carotid artery using DeepMed paradigm. The system demonstrated an accuracy of 90%
7	Savaş et al. (91) (2019)	SDL	The study presented a CNN-based model for carotid artery wall classification using a database of 501 US scans. The ground truth labels for classification were taken by two specialists. The classification accuracy, sensitivity, and specificity of the model were 89.1%, 89%, and 88%, respectively
8	Biswas et al. (81) (2020)	SDL	The study compared the patch-based solution in AI framework against the traditional (non-patch-based) method for segmentation of the walls of the ultrasound-based carotid artery. Post segmentation, the walls of the carotid artery was quantified jointly using plaque area (PA) and carotid IMT (cIMT). The system consisted of 250 carotid scans that give the cIMT error of 0.0935 mm (lowest of all previous methods) and PA error of 2.7939 mm2. The correlation coefficient’s between AI and GT for PA and cIMT was 0.89 and 0.99, respectively
9	Jamthikar et al. (13) (2020)	SDL	Following was the three main finding of the paper: (a) there were two pathways mainly affected the atherosclerotic process, resulting in heart injury, (b) calculators that uses carotid US images performed better than the conventional calculators, (c) for CVD risk assessment in routine practice for RA patients using SDL AI methods
10	Meshram et al. (83) (2020)	SDL	The study showed superior performance using UNet-based paradigm. For automated and semi-automated methods for plaque segmentation, the Dice-based performance metric yielded 0.48 and 0.83, respectively. Using the bounding box scheme with 5% error margin, the Dice was 0.79 and 0.80, respectively for UNet and dilated UNet paradigms
11	Jain et al. (90) (2020)	SDL	The study presented a technique for localization of carotid artery in transverse B-mode ultrasound scans. The authors implemented a fast region convolutional neural network (FRCNN) system that combines regional proposal network (RPN) and object class detection network (PCDN). Using the correct bounding box, the highest IOU score obtained was 99%. The experiments were done for different number of epochs such as 30, 200, and 2000. The accuracies were highest for 2000 epochs showing the values as 89.91%, 89.71%, and 89.36% for K= 2, 5, and 10 respectively
12	Groves et al. (106) (2020)	HDL	A comparison of mask R-CNN and UNet algorithms was carried out for automatically segmenting carotid artery (CA) and internal jugular vein (IJV). The mask RCNN produced a more accurate vascular segmentation and 3-D reconstruction of the vasculature. This yielded similar accuracy as the manually segmented CT scan. It enabled automatic analysis of the neck vasculature. A dataset consisted of 2439 images. The DICE scores generated for the CA and IJV were 0.90 and 0.88, respectively for masked R-CNN respectively. The DICE scores for UNet were 0.81 and 0.71, for the CA and IJV, respectively
13	Tsakanikas et al. (107) (2020)	HDL	The carotid vessel segmentation was implemented using UNet-based CNN algorithm. Carotid atrial tree provided better results while the plaque tissue helped, in early detection treatment and prevention of carotid diseases. The UNet and morphological active contour were combined in a repetitive manner for segmenting the outer wall and carotid lumen. Using the MR image series obtained from TAXINOMSIS study, the system created a 3D meshed model by carotid bifurcation and smaller branches. The system showed an accuracy of 99.1% for lumen area and 92.6% for perimeter. Such models were applicable for computational fluid dynamics simulations
14	Koktzoglou et al. (95) (2020)	SDL	The study introduced a strategy for optimizing the image quality in ungated 3T MRA for non-contrast extracranial carotid artery using DNN-based solution. The strategy took three minutes using single-shot radial sampling method, and it was benchmarked against 3-D filtering-based denoiser. The results of DL-based solution outperformed compared to 3D filtering method. Radial k-space sampling provided an increased arterial temporal signal-to-noise ratio (tSNR) by 107% and further improved image quality during 1-shot imaging. DL-based image processing gave a 24% and 195% increase when compared to original QISS score in arterial-to-background contrast and apparent contrast-to-noise ratio (aCNR)
15	Xiao et al. (97) (2020)	SDL (RF Signal)	The study presented a DL-based model for segmentation of carotid vessel wall using RF signals. This was shown to be useful for understanding the mechanical properties of the carotid arteries, which indirectly measures the CVD risk. The DL-based methods showed a higher accuracy in tracking the wall motion. The DL method was compared against the block matching strategy. The performance was evaluated by using the displacement estimation error in Z and X direction. For Z-direction, the distance estimation error was 94.8% (DL method), 93.2% (block matching method). For X-direction, the distance estimation error was 94.2% (DL method) and 92.9% (block matching method)
16	Biswas et al. (80) (2021)	SDL	The authors presented the review study that summarized the impact and evolution of cIMT/PA assessment using AI techniques. The authors compared four different segmentation techniques namely, traditional, semi-automated, ML, and DL-based. Further, ML/DL techniques were expressed in a mathematical way. The DL-based mechanisms are better due to automated feature extraction in the DL system
17	Jain et al. (98) (2021)	HDL	This study uses the “Unseen AI” technique i.e., the training and testing datasets are from different ethnic groups. A four-layered UNet architecture were used for segmentation. The wall plaque area was measured for evaluation. The PE results for “Unseen AI” pair one were mean accuracy, DICE similarity, and CC with values 98.55, 78.38, and 0.80, respectively. For “Unseen AI” pair two, the values were 98.67, 82.49, and 0.87, respectively. The “seen AI” gave 99.01, 86.89, and 0.92, respectively. The DL-based models were validated for low atherosclerotic wall plaque segmentation by hypothesizing that “Unseen AI” lies in a very close proximity to “Seen AI” having a difference was almost less than 10%. The running time for the online system was almost less than one second. These DL-based methods can be used for CVD risk stratification
18	Jain et al. (99) (2021)	HDL	The authors introduced the SegNet-UNet HDL architecture by applying it to a dataset with 970 ICA US images. Where, the SegNet was placed above the UNet for the HDL design. The performance were compared between five SDL/HDL architecture, namely UNet, UNet+, SegNet, SegNet-UNet, and SegNet-UNet+. The K10 model was applied in the study. The AUC values were 0.91, 0.911, 0.908, 0.905, and 0.898 (CE-loss) corresponding to UNet, UNet+, SegNet, SegNet-UNet, and SegNet-UNet+, respectively. Similarly, for DSC-loss the values were 0.883, 0.889, 0.905, 0.889, and 0.907, for the same order of models. The correlation values were 0.98, 0.96, 0.97, 0.98, and 0.97 for CE-loss models between AI-based PA and ground truth PA (GTPA). For DSC-loss models the values were 0.98, 0.98, 0.97, 0.98, and 0.98. It was concluded that SegNet-UNet outperformed other systems
19	Zhou et al. (85) (2021)	SDL	The study demonstrated the usage of DL paradigm to segment the plaque in carotid longitudinal ultrasound scans, which was later used for TPA measurement. The authors showed high accuracy, low variability and suitability for (a) monitoring the response to therapy and (b) investigating new therapeutic methods for clinical trials. The two UNet models (M1 and M2) were trained with two GTs corresponding to two different observers. The performance evaluation for M1 and M2 models using Pearson correlation coefficient showed the values as 0.989, 0.987. The mean TPA difference when compared between UNet and manual segmentation ranged from 4.7 mm2 to 312.8 mm2. The mean segmentation time was 8.3±3.1 ms
20	Zhou et al. (86) (2021)	SDL	The study presented DL paradigm for carotid plaque segmentation and TPA measurement, however, the study used a very small subset of data sets. A UNet++ ensemble algorithm was suggested in this study that used the 2D carotid US images. The gold standard used was a set of manually segmented images. The PE used consisted of the following metrics, namely, (I) Dice, (II) difference of area measurement (ΔTPA), (III) Pearson correlation coefficient, (IV) Bland-Altman plots, (V) intra-class correlation coefficient (ICC), and (VI) coefficient-of-variation. For SPARC dataset, DSC, ICC, CoV gave the values 83.3-85.7%, 0.996, and 7.54%, respectively, while for Zhongnan dataset, DICE and ICC was 88.6% and 0.985, respectively
21	Ganitidis et al. (87) (2021)	SDL	The authors have introduced an interpretable classification paradigm for the risk assessment and plaque stratification using carotid US scans. The sampling was implemented by applying an ensemble learning scheme to maintain the balance between the symptomatic and asymptomatic classes. The primary ensemble-based models were built by using CNNs. Further, a six-layer deep CNN was used for automatic feature extraction process, followed by two fully connected layers. The DL system showed the three performance metrics such as area-under-the-ROC curve (AUC), sensitivity, and specificity having the values of 73%, 75% and 70%, respectively. The authors concluded that the clinical relevance was that the combination of DL paradigm with the interpretability methods and this facilitated the risk stratification process
22	Mohannadi et al. (100) (2021)	HDL	The study adopted a DL-based technique that can be applied to perform semantic segmentation of intima-media complex, and further calculating the cIMT measurement. A fully automated encoder-decoder based prototype was used to overcome the lack of large size datasets as it used multi-image inputs providing a good learning to the system with different features. The encoder and decoder had stages namely, convolution, batch normalization, and parametric ReLU (PReLU). The performance evaluation was done by measuring F1-measure, precision, recall, DICE coefficient, JI having values of 79.92%, 81.18%, 82.06%, 74.23%, and 60.24%, respectively. From the obtained results, the authors concluded that the proposed encoder-decoder architecture out performed all other state-of-the-art work
23	Latha et al. (101) (2021)	HDL	The authors used US images of carotid artery for IMT and plaque diameter. ML and DL approaches were used for recognizing symptomatic or asymptomatic plaque in a total of 361 images (202 normal images and 159 images with carotid plaque). The ML-based algorithm used were CART DT, RF, and logistic regression (LR), while DL-based models used were CNN, MobileNet, and CapsuleNet. The classification accuracy of CapsuleNet TL was 96.7%, unlike RF gave and accuracy of 91.41%
24	Otgonbaatar et al. (102) (2021)	HDL	The study was an application in CVD risk stratification using CT angiography. Compared to filtered-back projection and hybrid iterative reconstruction, the DL-based reconstruction was superior demonstrating small vessels. This helped in blooming artifact reduction thereby improving the image quality
25	Jain et al. (103) (2021)	HDL	The study adopted a DL-based solution using UNet and SegNet-UNet, while keeping the objective of speed, accuracy, and reliability during early detection and quantification of plaque lesions in CCA ultrasound scans. The system was benchmarked against AtheroEdge™ 2.0, demonstrating the accuracies of 93%, 94%, and 95%, respectively, corresponding to UNet, SegNet-UNet, and AtheroEdge™ 2.0 systems
26	Ziegler et al. (88) (2021)	SDL	The study showed the usage of SDL model based on UNet, keeping the spirit of branch-level segmentation of carotid arteries that benefitted large-cohort investigations. The performance evaluation yielded Dice, Mathew, and true positive ratio of 0.80, 0.80, and 0.84, respectively
27	Bortsova et al. (89) (2021)	SDL	The study used automated UNet-based DL solution for segmentation and intracranial carotid artery calcification (ICAC) volume measurement. While comparing against the manual methods, the sensitivity and PPV were 83.3%, and 88%, respectively. The authors showed the correlation between ICAC and incident stroke
28	Zhu et al. (104) (2021)	HDL	The study used 3D residual-UNet DL approach for segmentation of lumen and wall in a diseased carotid artery. The identity mapping was done by using an optimal channel fitting structure. The strategies used for training the MRimages are patch-level and global level. Optimization was done to the pre-segmentation results, later cascaded with the patch-croped MRvolume data and trained for segmenting the carotid lumen and wall. The segmentation was reproduceable and showed the Dice of 0.84 and 0.74 for lumen and wall, respectively
29	Wasih et al. (92) (2021)	SDL	The study presented two sets of models, namely automated RCNN and UNet for segmentation carotid artery, internal jugular vein from the transverse US scans of neck. The RCNN model was used for mask generation while UNet model was used for selection of the largest connected region for each class. The US models were validated using CT-based imaging. The performance was evaluated using Dice score which came out to be within two mm between US and CT
30	Flores et al. (108) (2021)	HDL	This study presented a review using MI and AI for peripheral artery disease (PAD). Finally, the study discusses the potential areas for the future of PAD care and advanced solution such as analytics. The DL reconstruction techniques improves the image quality of brain CT angiography. The objective measurement and the subjective grading got improved when compared with filtered-back-projection and hybrid iterative reconstructions
31	Luo et al. (96) (2021)	SDL (Doppler US)	The study developed a DL-based model to classify aortoiliac, femoropopliteal, and trifurcation disease in the US studies. This was then benchmarked against RF-based ML algorithm for classification of carotid stenosis in duplex US. The experienced physician was used for gold standard readings. The NN model used waveforms, pressures values, flow velocities, and plaque presence. AI was developed to automate the interpretation of these LEAD and carotid duplex ultrasound studies. The DL model obtained the performances in the form of accuracy of prediction of normal, aortoiliac disease, femoropopliteal disease, and trifurcation disease as 97%, 88.2%, 90.1%, and 90.5%, respectively. For internal carotid artery stenosis, the accuracies were classified as per the stenosis range 0-49%, 50-69%, > 70%, and 100% occlusion, having the accuracies of 99.2%, 100%, 100%, and 100%, respectively
32	Rim et al. (166) (2021)	SDL (CAC Score)	The study showed that DL-based retinal photograph-derived CAC score that can be used as an alternative to CT scan-measured CAC in evaluating the cardiovascular events. The system RetiCAC showed a superior performance resulting an AUC of 0.742, when compared to single parameter models (Age: 0,705, glucose: 0.637)
33	Park et al. (105) (2022)	HDL	The study proposed DL-UFV neural network-based on segmentation. The system combined the speckle tracking velocimetry and the speckle image velocimetry for vessel wall segmentation. The parameters measured were vascular stiffness and velocity field information of blood flow. The system improved biases in measurements of velocity, wall shear stress (WSS), and strain by 4.6-fold, 115.1-fold, and 22.2-fold, respectively.
34	Jain et al. (161) (2022)	HDL	The study designed HDL models which was then benchmarked against the conventional SDL models. The HDL designed were Inception-UNet, Squeeze-UNet, and Fractal-UNet. The benchmarked SDL models were UNet, UNet+, UNet++ and UNet+++. The HDL models showed low memory, faster operations, and small training time of parameters. The coefficient of correlation metric provided 0.96, 0.96, 0.98, 0.95, 0.96, and 0.96 for CCA for seven SDL and HDL models respectively, whereas ICA resulted 0.99, 0.99, 0.98, 0.99, 0.98, 0.98 and 0.98 respectively. AUC for CCA images were 0.97, 0.969. 0.974, 0.969, 0.962, and 0.960 respectively, while for ICA images were 0.99, 0.989, 0.988, 0.989, 0.986, 0.989, and 0.988, respectively (P<0.001)

Table 3. Characteristics of deep learning systems for CVD risk predictions (I).

SN	Studies	Country	TP	TI	FH	RF	BMI	Eth	#TD	HT	SM	MC	MRI	ECG	CUSIP	IST	Tech.	Organ
1	Azzorpardi et al. (78)	England	15	×	×	CUSIP	×	√	PD	√	√	√	×	×	√	Image	Segm.	Carotid Artery
2	Biswas et al. (79)	USA	203	396	√	OBBM, LBBM, cIMT	√	√	ID	√	√	√	×	×	√	Image	Segm.	Carotid Artery
3	Biswas et al. (80)	USA	203	×	×	cIMT, PA	√	√	ID	√	√	×	×	×	√	Image	Segm.	Carotid Artery
4	Biswas et al. (81)	USA	204	×	×	OBBM, LBBM, cIMT, TPA	√	√	ID	√	√	√	×	×	√	Image	Segm.	Carotid Artery
5	Jain et al. (98)	USA	50	300	√	OBBM, LBBM, CUSIP	√	√	ID	√	√	√	×	×	√	Image	Segm.	Carotid Artery
6	Jain et al. (99)	USA	99	970	×	OBBM, LBBM, CUSIP	×	√	ID	√	√	√	×	×	√	Image	Segm.	Carotid Artery
7	Jamthikar et al. (13)	USA	120	×	√	OBBM, LBBM, CUSIP, RA	√	√	PD	√	√	×	×	×	√	Image	Segm.	Carotid Artery
8	Lakadir et al. (82)	Spain	56	×	√	OBBM, LBBM, cIMT	√	×	ID	√	√	×	×	×	√	Image	Segm.	Carotid Plaque
9	Meshram et al. (83)	Columbia	101	×	√	OBBM, LBBM, Plaque	√	×	ID	√	√	×	×	×	√	Image	Segm.	Carotid Plaque
10	Wu et al. (84)	China	1,057	×	√	OBBM, LBBM, MRI	√	×	ID	√	√	×	√	×	×	Image	Segm.	Carotid
11	Zhou et al. (85)	China	600	5,000	×	OBBM, LBBM, TPA	×	√	ID	×	×	√	×	×	√	Image	Segm.	Carotid Plaque
12	Zhou et al. (86)	China	77,497	×	×	OBBM, LBBM, TPA	×	√	ID	√	√	√	×	×	√	Image	Segm.	Carotid Plaque
13	Ganitidis et al. (87)	Greece	84	×	×	OBBM, LBBM, TPA	×	√	ID	√	√	×	×	×	√	Image	Segm.	Carotid
14	Mohannadi et al. (100)	Qutar	NA	100	×	OBBM, cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
15	Latha et al. (101)	Malaysia	NA	361	√	OBBM, LBBM, cIMT	√	×	ID	√	√	×	×	×	√	Image	Segm.	Carotid
16	Otgonbaatar et al. (102)	Korea	43	×	×	OBBM, Brain CT	×	×	ID	√	√	×	×	×	√	Image	Segm.	Brain
17	Jain et al. (103)	USA	190	379	√	OBBM, LBBM, CT	√	√	ID	√	√	×	×	×	√	Image	Segm.	Carotid
18	Ziegler et al. (88)	Sweden	NA	482	√	OBBM, LBBM, MRI	√	×	ID	√	√	×	√	×	×	Image	Segm.	Carotid
19	Bortsova et al. (89)	The Netherlands	2,319	×	√	OBBM, LBBM, volume	√	√	ID	√	√	×	×	×	√	Image	Segm.	Carotid
20	Zhu et al. (104)	China	NA	×	×	OBBM, LBBM, cIMT	×	×	ID	×	×	×	√	×	×	Image	Segm.	Carotid
21	Park et al. (105)	Korea	NA	316	×	US Imaging parameter	×	√	ID	×	×	×	×	×	√	Image	Segm.	Carotid
22	Jain et al. (90)	USA	NA	433	√	cIMT, LD	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
23	Savaş et al. (91)	Turkey	153	501	×	OBBM, IMT	×	×	ID	√	×	×	×	×	√	Image	Segm.	Carotid
24	Washim et al. (92)	USA	NA	20	×	cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid Artery
25	Sudha et al. (93)	India	110	220	×	cIMT	×	×	ID	√	×	√	×	×	√	Image	Segm.	Carotid
26	Groves et al. (106)	Canada	160	360	×	OBBM, LBBM, TPA	×	×	ID	√	×	√	×	×	√	Image	Segm.	Carotid
27	Saba et al. (94)	Italy	75	×	×	cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
28	Tsakanikas et al. (107)	Greece	30	×	√	cIMT	×	×	ID	×	×	×	√	×	×	Image	Segm.	Carotid
29	Koktzoglou et al. (95)	Illinois	12	×	×	cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
30	Flores et al. (108)	USA	NA	×	×	cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
31	Luo et al. (96)	Indianapolis	5700	×	×	cIMT	×	×	ID	×	×	×	×	×	√	Image	Segm.	Carotid
32	Xiao et al. (97)	China	NA	×	×	RF Signals	×	×	ID	×	×	×	×	×	×	RFsignal	Segm.	Carotid

CVD, cardiovascular disease; SN, serial number; TP, total patients; TI, total image; FH, family history; RF, risk factor; BMI, body mass index; Eth, ethnicity; #TD, #type of data; HT, hypertension; SM, smoking; MC, multicentre; MRI, magnetic resonance imaging; ECG, electrocardiogram; CUSIP, carotid ultrasound imaging phenotype; IST, input single type; Tech., technique; CUSIP, carotid ultrasound image phenotypes; OBBM, office-based biomarkers; LBBM, laboratory-based biomarkers; cIMT, carotid intima-media thickness; PA, power analysis; TPA, total plaque area; RA, rheumatoid arteries; CT, computed tomography; US, ultrasound; MRI, magnetic resonance imaging; LD, lumen diameter; RF, random forest.

Table 4. Characteristics of deep learning systems for CVD risk predictions (II).

SN	Studies	# GT	GT N	# AU	DL	# ML C	CT	FE	HID	HU	LU	Protocol
1	Azzorpardi et al. (78)	2	CVD, non-CVD	1	UNet	×	×	√	√	×	4	15
2	Biswas et al. (79)	2	CVD, non-CVD	1	CNN	×	×	√	√	×	4	10
3	Biswas et al. (80)	2	CVD, non-CVD	1	ANN	×	×	√	√	×	3	1
4	Biswas et al. (81)	2	CVD, non-CVD	1	CNN, FNN	×	×	√	√	√	10	10
5	Jain et al. (98)	2	CVD, non-CVD	1	UNet	×	×	√	√	×	4	10
6	Jain et al. (99)	2	CVD, non-CVD	1	HDL, SDL	×	×	√	√	×	3	10
7	Jamthikar et al. (13)	3	CVD, non-CVD, RA	1	DL	√	SVM	√	√	×	1	×
8	Lakadir et al. (82)	2	CVD, non-CVD	1	CNN	√	SVM	√	√	×	9	5
9	Meshram et al. (83)	2	CVD, non-CVD	1	UNet	×	×	√	√	×	4	×
10	Wu et al. (84)	2	CVD, non-CVD	1	DeepMAD	×	×	√	×	×	3	×
11	Zhou et al. (85)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	MCar*
12	Zhou et al. (86)	2	CVD, non-CVD	1	Unet++, CNN	×	×	√	√	×	4	×
13	Ganitidis et al. (87)	2	CVD, non-CVD	1	CNN	×	×	√	√	×	4	4
14	Mohannadi et al. (100)	2	CVD, non-CVD	1	CNN, Unet	×	×	√	×	√	6	×
15	Latha et al. (101)	2	CVD, non-CVD	1	CNN	×	×	√	×	√	3	×
16	Otgonbaatar et al. (102)	2	Cerebrovascular Disease	3	FBP, HIR, DLR	×	×	√	×	×	3	×
17	Jain et al. (103)	2	CVD, non-CVD	2	SegNet-Unet	×	×	√	×	√	224	10
18	Ziegler et al. (88)	2	CVD, non-CVD	1	CNN, DM	×	×	√	×	×	11	10
19	Bortsova et al. (89)	2	CVD, non-CVD	1	DeepEnsemble	×	×	√	×	√	4	10
20	Zhu et al. (104)	2	CVD, non-CVD	1	Unet++	×	×	√	×	√	2	5
21	Park et al. (105)	2	CVD, non-CVD	1	CNN	×	×	√	×	√	×	10
22	Jain et al. (90)	2	CVD, non-CVD	1	FNCNN	×	×	√	×	√	3	2, 5, 10
23	Savaş et al. (91)	2	CVD, non-CVD	2	DNN, ANN	×	×	√	×	1	3	×
24	Washim et al. (92)	2	CVD, non-CVD	1	CFNet	×	×	√	×	×	5	×
25	Sudha et al. (93)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	×
26	Groves et al. (106)	2	CVD, non-CVD	1	R-CNN	×	×	√	×	×	2	4
27	Saba et al. (94)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	×
28	Tsakanikas et al. (107)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	×
29	Koktzoglou et al. (95)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	×
30	Flores et al. (108)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	5,10
31	Luo et al. (96)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	×
32	Xiao et al. (97)	2	CVD, non-CVD	1	CNN	×	×	√	×	×	4	4

CVD, cardiovascular disease; SN, serial number; # GT, ground truth; GT N, GT name; # AU, number of algorithm used; DL, deep learning; # ML C, number of ML classifier; CT, classifier type; FE, feature extraction; HID, handling imbalanced data; HU, hyperparameters used; LU, layers used.

Table 5. Characteristics of deep learning systems for CVD risk predictions (III).

SN	Studies	#PE P	Sen	Spec	Acc	PPV	NPV	FPR	Pre	F1 Score	P value	HL	D Coff.	JI
1	Azzorpardi et al. (78)	3	√	√	×	×	×	×	×	×	√	×	√	×
2	Biswas et al. (79)	3	×	×	√	×	×	×	×	×	√	×	√	×
3	Biswas et al. (80)	4	√	√	95.4	×	×	×	×	×	×	×	√	×
4	Biswas et al. (81)	3	O	O	99	×	×	×	×	×	√	×	√	×
5	Jain et al. (98)	7	√	√	√	×	×	×	√	×	√	×	√	√
6	Jain et al. (99)	7	√	√	√	×	×	×	√	×	√	×	√	√
7	Jamthikar et al. (13)	3	√	√	√	×	×	×	×	×	×	×	×	×
8	Lakadir et al. (82)	3	83	90	78.5	×	×	×	×	×	×	×	×	×
9	Meshram et al. (83)	3	83	√	O	×	×	×	×	×	×	×	√	×
10	Wu et al. (84)	3	√	√	89	×	×	×	×	×	×	×	√	×
11	Zhou et al. (85)	5	√	√	√	×	×	×	√	×	√	×	×	×
12	Zhou et al. (86)	6	√	√	√	×	×	×	√	×	√	×	√	×
13	Ganitidis et al. (87)	3	75	70	75	×	×	×	×	×	×	×	×	×
14	Mohannadi et al. (100)	3	√	√	98	×	×	×	×	×	×	×	√	×
15	Latha et al. (101)	5	√	√	100	×	×	×	×	×	×	×	√	√
16	Otgonbaatar et al. (102)	3	√	√	√	×	×	×	×	×	×	×	×	×
17	Jain et al. (103)	6	√	√	√	×	×	×	√	×	×	×	√	√
18	Ziegler et al. (88)	7	√	√	√	×	×	×	√	×	√	×	√	√
19	Bortsova et al. (89)	3	83.8	×	×	88	×	×	×	×	√	×	×	×
20	Zhu et al. (104)	3	×	×	×	×	×	×	×	×	×	√	√	√
21	Park et al. (105)	3	√	√	√	×	×	×	×	×	×	×	×	×
22	Jain et al. (90)	4	×	×	√	×	×	×	×	×	×	√	√	√
23	Savaş et al. (91)	6	×	×	√	√	√	√	√	√	×	×	×	×
24	Washim et al. (92)	1	×	×	√	×	×	×	×	×	×	×	×	×
25	Sudha et al. (93)	2	√	×	×	×	×	×	×	×	×	√	×	×
26	Groves et al. (106)	4	×	×	√	×	√	√	×	×	×	√	×	×
27	Saba et al. (94)	3	×	×	√	×	×	√	×	×	×	×	×	√
28	Tsakanikas et al. (107)	6	×	×	√	√	×	×	×	×	×	×	×	√
29	Koktzoglou et al. (95)	6	√	×	√	×	×	√	√	×	√	×	×	√
30	Flores et al. (108)	6	√	√	√	√	×	×	×	√	√	×	×	×
31	Luo et al. (96)	1	×	×	√	×	×	×	×	×	×	×	×	×
32	Xiao et al. (97)	3	√	×	√	√	×	×	×	×	×	×	×	×

CVD, cardiovascular disease; SN, serial number; # PE P, number of PE parameters; Sen, sensitivity; Spec, specificity; Acc, accuracy; PPV, positive predictive value; NPV, negative predictive value; FPR, false positive rate; Pre, precision; HL, Hamming loss; D Coff., DICE coefficient; JI, Jaccard-index.

Table 6. Characteristics of deep learning systems for CVD risk predictions (IV).

SN	Studies	SA	PA	V	HA	P TT	MWT	WT	KWT	DOR
1	Azzorpardi et al. (78)	√	×	√	×	×	×	×	×	×
2	Biswas et al. (79)	√	×	√	×	√	√	√	√	×
3	Biswas et al. (80)	√	×	√	×	×	×	×	×	√
4	Biswas et al. (81)	√	×	√	×	×	×	×	×	×
5	Jain et al. (98)	√	×	√	×	×	×	×	×	×
6	Jain et al. (99)	√	×	√	×	×	×	×	×	×
7	Jamthikar et al. (13)	√	×	√	×	×	×	×	×	×
8	Lakadir et al. (82)	√	×	√	×	×	×	×	×	×
9	Meshram et al. (83)	√	×	√	×	×	×	×	×	×
10	Wu et al. (84)	√	×	√	×	×	×	×	×	×
11	Zhou et al. (85)	√	×	√	×	×	×	×	×	×
12	Zhou et al. (86)	√	×	√	×	×	×	×	×	×
13	Ganitidis et al. (87)	√	×	√	×	×	×	×	×	×
14	Mohannadi et al. (100)	√	×	√	×	×	×	×	×	×
15	Latha et al. (101)	√	×	√	×	×	×	×	×	×
16	Otgonbaatar et al. (102)	√	×	√	×	×	×	×	×	×
17	Jain et al. (103)	√	×	√	×	×	×	×	×	×
18	Ziegler et al. (88)	√	×	√	×	×	×	√	×	×
19	Bortsova et al. (89)	√	×	√	√	×	×	×	×	×
20	Zhu et al. (104)	√	×	√	×	×	×	×	×	×
21	Park et al. (105)	√	×	√	×	×	×	×	×	×
22	Jain et al. (90)	√	×	√	×	×	×	×	×	×
23	Savaş et al. (91)	√	×	×	×	×	×	×	×	×
24	Washim et al. (92)	√	×	√	×	×	×	×	×	×
25	Sudha et al. (93)	√	×	√	×	×	×	×	×	×
26	Groves et al. (106)	√	×	√	×	×	×	×	×	×
27	Saba et al. (94)	√	×	×	×	×	×	×	×	×
28	Tsakanikas et al. (107)	√	×	×	×	×	×	×	×	×
29	Koktzoglou et al. (95)	√	×	×	×	×	×	×	×	×
30	Flores et al. (108)	√	×	×	×	×	×	×	×	×
31	Luo et al. (96)	√	×	×	×	×	×	×	×	×
32	Xiao et al. (97)	√	×	×	×	×	×	×	×	×

CVD, cardiovascular disease; SN, serial number; SA, statistical analysis; PA, power analysis; V, validation; HA, Hazard analysis; P TT, Paired t-test; MWT, Mann-Whitney test; WT, Wilcoxon test; KWT, Kruskal–Wallis test; DOR, diagnostic odds ratio.

The number of patients used in different studies ranges from 15 to 77,497, whereas the total number of images used varies from 20 to 5,000. The demographic factors considered by most of the studies were family history, and body mass index (BMI). Risk factors that were utilized in the DL-based systems were OBBM, LBBM, CUSIP, cIMT, TPA, PA, and LD. The mostly used DL architecture is a CNN, followed by UNet, UNet++, DeepMad, RCNN, deep ensemble, etc. The performance evaluation parameters used were accuracy, sensitivity, specificity, the AUC, F1-score, and Jaccard Index. The techniques used were the segmentation of images and risk of class classifications. Two types of data (Table 3) were used for input namely, image and point data.

Cardiovascular risk assessment via carotid plaque characterization: brief overview

The morphology of plaque plays a vital role in CVD risk prediction. The process of evaluating the atherosclerotic plaque characteristics using the grayscale tones falls under tissue characteristics (167-169). Tissue characterization has been applied to several fields of medical imaging such as stroke (135,170-172), plaque (173,174), coronary artery disease (175), retinal (176), cancer such as skin (177), thyroid (136), liver (44), ovarian (178), and prostate (179). Such classification can also be applicable to non-image-based characterization, such as diabetes (180), gene analysis (181), etc.

There are basically two types of plaque (I) symptomatic or (II) asymptomatic (Figure 8) (145,182,183). The symptomatic plaque is hypoechoic (dark) in nature. This is due to the presence of a large lipid core and minimal collagen. Further, the plaque is more heterogeneous in makeup and is therefore considered dangerous (unstable). On the contrary, the asymptomatic plaque is hyperechoic (bright) due to its small lipid core, abundant collagen, and the fact that it is often calcified. Therefore, this plaque is less dangerous (stable). The characterization of plaque can be done by using several imaging techniques namely, ultrasound (173), MRI (184), and CT (185) imaging. Carotid ultrasound is more frequently used for imaging of the carotid plaque (186-188). The CVD risk can be classified into low, medium, and high-based symptomatic asymptomatic carotid index (SACI) which is obtained from an ultrasound (US) scan. A high SACI value suggests that the image shows symptomatic, and a low value indicates asymptomatic plaques. Different AI-based techniques have been used for the classification of plaques, namely, ML-based or DL-based (189).

Figure 8.

Comparison between (A) symptomatic and (B) asymptomatic plaque (Courtesy of AtheroPoint™, CA, USA,) (Open Access).

Two-generation have evolved namely, Atheromatic™ 1.0 (128,190,191) and Atheromatic™ 2.0 (192) (AtheroPoint™, Roseville, CA, USA) for ultrasound-based plaque tissue characterization. Atheromatic™ 1.0 is based on ML-based techniques whereas Atheromatic™ 2.0 is based on DL-based techniques. In Atheromatic™ 1.0, the training features are extracted, then using the classifiers (namely, SVM, RF, XGBoost, NB, K-nearest neighbour (KNN) and the gold standard, the training models were developed offline. These models were then used to transform the testing features to yield the predicted labels (symptomatic vs. asymptomatic).

Atheromatic 2.0 uses DL mechanisms like Deep convolutional neural network (DCNN), Visual geometric group-16 (VGG16), and transfer learning (TL) (193) for feature selection and then classifying the plaque (103). The DL techniques can be of different types like SDL and HDL (99,194). HDL uses the fusion of two SDL models or the fusion of SDL with ML classifiers. All types of DL paradigms provide automated feature extraction using convolutional filters followed by max pooling. The process is repeated for extracting the best features. Since ML systems are not included in the scope of current work, they will not be discussed here. Thus, DL techniques for plaque classifications and CVD risk assessment can provide us with a non-invasive, accurate, and economic framework benefiting the patient cost (Figure 9). ML or DL systems sometimes (I) overrate the accuracy with a lack of clinical validation, (II) lack data augmentation in the unbalanced data classes during the DL training, (III) lack of scientific validation of architectures leads to AI bias. These will be discussed in the next section.

Figure 9.

General architecture for plaque tissue characterization and classification system (Courtesy of AtheroPoint™, CA, USA).

Cardiovascular disease risk assessment in COVID-19 framework

COVID-19 is in its third year since it started in December 2019 and has infected more than 660 million people, killing about seven million people (195). It is a very pathogenic viral infection and is highly transmissible caused due to a high rate of acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (7). The individual’s lung gets affected due to COVID-19 (196-199) and all the other organs get affected in different ways and degrees (200-205). Also, by now it is very clear that the mortality in COVID-19 is because of viral pneumonia triggered by acute respiratory distress syndrome (ARDS) that damages the lungs and pulmonary arteries. The COVID-19 virus enters the lung through aerosol transmission, and afterward gets associated with the host cells leading to several respiratory symptoms. And eventually, the risk of CVD gets worse with an increase in comorbidities (203,206). COVID-19 affects the lung and then the heart/brain eventually through different pathways as shown in Figure 10). CVD gets triggered as the angiotensin-converting enzyme 2 (ACE2), a host cell receptor as the viral point protein increases and directly affects the heart. This process of atherosclerosis gets accelerated during the COVID, generally known as cardiomyopathy (199,207). The early assessment of CVD becomes more important during COVID as the disease gets accelerated. The assessment and validation can be performed using several imaging techniques (208-212). The CVD can be predicted by atrial imaging (Figure 11). The process has mainly two types of screening namely, (I) primary, and (II) secondary for diagnosis of CVD. The primary screening involves screening with the help of robots along with AI. This basically includes questioning the patient. Based on the answers, further decisions for different test requirements are made by the robot and AI. Based on the test result whether it is positive or negative, further CVD risk assessment is conducted. The secondary screening is based on CVD risk prediction. Ground glass opacities (GGO) of the patient need to be analyzed for finalizing the period of quarantine for the patient. Thus, CVD risk stratification can be done by use of different AI approaches like ML and DL. AtheroEdge™ 3.0 is such an application that uses the DL approach for CVD risk estimation from the carotid imaging of the individual having COVID-19 (99).

Figure 10.

ARDS formation due to COVID-19 (191-196) (A) and CVD risk stratification (B). The arrows indicate the “Far wall of the carotid artery showing the plaque burden”. The red line is the lumen-intima (LI) border and the green line is the media-adventitia (MA) border. ARDS, acute respiratory distress syndrome; COVID-19, Corona Virus Disease 2019; CVD, cardiovascular disease. ACE2, angiotensin-converting enzyme 2; eGFR, estimated glomerular filtration rate; ESR, erythrocyte sedimentation rate; BP, blood pressure; BMI, body mass index.

Figure 11.

Integration of Arterial Imaging in COVID-19 framework (Courtesy of AtheroPoint™, CA, USA) (200-205) (A). AIbased CVD risk stratification using a fusion of OBBM, LBBM, CUSIP, and MedUSE (Courtesy of AtheroPoint™, CA, USA) (B). COVID-19, Corona Virus Disease 2019; CVD, cardiovascular disease; LBBM, laboratory-based biomarkers; OBBM, office-based biomarkers; CUSIP, carotid ultrasound image phenotype.

ML is used for the prediction of CVD risk by importing imaging data from the vascular level for CVD risk prediction. The ML system acquires the plaque image-based phenotypes from the scans of the carotids of the patient, and the risk was predicted for CVD. The main component of the AI system is the training model shown in green color as imaging-based AI along with AI and robot logo. The ML output suggests to the clinician if the CVD risk is low, moderate, or high the monitoring patient (M1). The predicted CVD risk is classified based on the color-coded scheme (Figure 10). This system includes the GGO evaluation of the CT lungs or any pulmonary embolism (138,213-218). The interface is shown in Figure 11. Recently there have been more advances in AI for risk prediction in CVD in COVID-19 paradigms (219). We can conclude that AI is playing a vital role in CVD risk stratification during the COVID-19 times. The next section shows discuss the principal findings, benchmarking, non-linearity, and the future of CVD risk stratification.

Cardiovascular risk stratification using deep learning-based ECG

Another way for the CVD risk assessment was by using the ECG signals as one of the risk factors in the input along with the different gold standards (220-228). Different phenotypes gained from the ECG signals such as RR interval, length, and heartbeats were used as risk factors. The different DL-based methods when used with ECG in comparison to using with the OBBM, LBBM, and image phenotypes as input covariates. These modifications were required as ECG signals are heterogeneous in nature containing beats, different heart conditions like abnormal heart rhythms, aorta disease, congenital heart disease, and CAD and sequence patterns. The modified UNet architecture is shown in Figure 12. The UNet architecture shows the ECG signals were being classified into the R-peaks where the R-peaks were detected, confidence maps consisting of left bundle branch block (LBBB) and premature ventricular contractions (PVC) were identified. The modification helped the system to handle the heterogeneity and automatic extraction of features from the ECG. The main modification required was the introduction of multiple classification heads for R peak detection, and mapping out conditions. The same convolutional operation was applied to obtain the feature maps with equal sizes algorithms used for CVD risk stratification, which were CNN (229,230), UNet (231-233), and long short-term. The general kernel size used was 3×1. Three compression stages were applied, where each feature map was reduced and the number of features were doubled. The global average pooling layer was used while training so that the confidence maps don’t converge any non-existent class within the segment. The PE parameters used in the ECG-based DL systems were confusion metrics (229,230), accuracy (234,235), sensitivity (231-233), specificity (236,237), PPV for justifying the reliability of the systems (238,239). Recently there have been more advances in AI for CVD risk prediction using ECG-based DL systems (240-243).

Figure 12.

Modified UNet architecture for ECG-based DL systems for CVD risk stratification. ECG, electrocardiogram; DL, deep learning; CVD, cardiovascular disease.

Artificial intelligence explainability in cardiovascular disease

In the field of CVD risk stratification that requires image segmentation, DL has played a very important role. Our observations has shown that SDL and HDL offer stronger innovations and in particular, HDL offers superior performance against conventional models for segmentation and classification (244). One of the challenges in the DL models is its black box nature, even though, the design and performance meets the requirements and objectives. This black box nature has unaddressed “Wh’ questions such as what, why, and how the DL output can be interpreted. More important questions are how will prediction change if the training model changes, what will happen if the training parameters changes, how do prediction change if the feature changes, what are the limits on the features if the prediction needs to be same, etc. Such kind of challenges can be categorized as the subfield of AI, namely “explainable AI (XAI)” (245-251) (Figure 13). Our observations show that in the field of segmentation for CVD, there has been very limited number of studies using XAI. The role of XAI is even more important when dealing with CVD risk stratification having different kinds of clinical outcomes. The reasons why XAI has not penetrated in CVD is (I) the advances of XAI had recently started (2015) and (II) some of the XAI packages like Shapley additive explanations (SHAP), uniform manifold approximation and projection for dimension reduction (UMAP), GradCAM has not been integrated in DL packages.

Figure 13.

Different aspects of XAI for DL studies. AOG, And-Or graph; SVM, support vector machine; CNN, convolutional neural network; DBNs, deep belief networks; HDL, hybrid deep learning; XAI, Explainable AI; DL, deep learning.

Local interpretable model-agnostic explanations (LIME) (252) and SHAPLEY (253) were used for the point data while the heatmaps were for the image data. As per European general data protection regulation (GDPR) (Figure 14, right) (254), the important attributes such as privacy, transparency (255), fairness, and explainability (254,256) (Figure 14, left) were highlighted for the design of ML (257-261) and DL paradigm.

Figure 14.

Left: Trustable AI or Trustworthy AI includes Valid AI, Responsible AI, Privacy-Preserving AI, and XAI. Right: EU General Data Protection Regulation (GDPR) (Open Access). XAI, Explainable AI.

Since XAI incorporates the feedback loop, the customized eight steps of DL are mainly (I) DL training, (II) quality assurance (QA), (III) installation/deployment, (IV) prediction, (V) cross-validation-based testing (A/B test), (VI) monitoring, and (VII) debugging, and (VIII) feedback loop (262). Some specific guidelines were allowed to maintain the above-mentioned explainability (263). Figure 14, left below displays all eight steps.

Artificial intelligence pruning in cardiovascular disease risk stratification

It creates a challenge to perform real-time processing of the medical images. Some alternatives were available in the market like graphical processing units, supercomputers, etc. which are expensive for general use and out of reach. The DL systems undergoes heavyweights processing in the different layers of neural networks, along with a high number of epochs and training iterations per epoch.

As an alternative, pruning is an optimization technique used for eliminating the unlinked weights which are the nodes of the neural network that has less impact on the neural network performance. This helps in improving the training model storage and speed. Evolutionary algorithms exist which can be used for compressing the DL models, namely (I) particle swarm optimization algorithm (PSO), (II) differential evolution algorithm (DEA), (III) whale optimization (WO) algorithm, and (IV) genetic algorithm (GA) (264,265). These evolutionary optimization techniques were embedded in the DL paradigm for fulfilling the segmenting objectives. Such DL systems are fully connected networks (FCN) or SegNet models. There are three pruning methods which are typically used for pruning the neural networks can be studied in more detail and are named as: (I) channel pruning (266-275), (II) hybrid pruning (276-278), and (III) weight or network pruning (279-284).

Risk of Bias estimation in deep learning-based cardiovascular disease systems

DL strategies have started to penetrate the CVD field, more so often SDL-based methods compared to HDL-based, which is still in its infancy stage. Even though DL-based systems provide a better results as compared to ML-based techniques because of their automated feature extraction paradigm, there still exists RoB. Five methods were used for bias estimation, namely, (I) ranking method (RBS), (II) region-based map (RBM), (III) radial bias area (RBA), (IV) prediction model risk of bias assessment tool (PROBAST), and (V) risk of bias in non-randomized studies-of interventions (ROBINS-I), as discussed in subsections “Risk of bias via ranking method”, “Radial-bias map method”, “Radial-bias area method”, “ROBINS-I”, and “PROBAST” respectively. For ranking method, we have merged a few AI attributes to obtain some meaningful inference. The final number of AI attributes considered for the ranking analysis was 29 (A1-A29). A comparative analysis was also performed which was mentioned in subsection “Venn diagram for analysing the five bias strategies”.

Risk of bias via ranking method

The RoB for the DL-based systems was estimated by using the ranking method (see Table S1) which depends on means values of the AI attributes and values of cumulative mean of all studies. These were calculated by scoring the corresponding attributes using a grading system (62,285). In total, twenty-nine attributes were selected for scoring the DL-based systems corresponding to demographic, DL architecture, performance evaluation, validation, and clinical evaluation. All the 32 DL studies were grouped as low-bias (6 studies), moderate-bias (11 studies), and high-bias (15 studies) based on the two types of cut-offs namely, low-moderate (LM) and moderate-high (MH) cut-offs. The values for LM cut-off was 3.3 while MH cut-off was 2.5 (Figure 15). The factors responsible for high-bias were not involvement of family history, body mass index (BMI), non-FDA approval, and lack of implementation in the clinical settings.

Figure 15.

Cumulative plot for all the DL-based CVD studies. LM, low-moderate; HM, moderate-high; DL, deep learning; CVD, cardiovascular disease.

Radial-bias map method

The concept behind the radial-bias map (RBM) stems by pictorially representing the strength of the AI attributes in the form of radius. If the end-points of the radius is spline fitted using a Bezier spline curve, then such a curve would represent the map of the bias, so-called “radial-bias map”. These radial lines would be angularly segregated by an angle of 12.4 (~360/29) degrees, where the denominator consisted of total number of AI attributes for each study. Since there were 29 attributes, we therefore clustered these AI attributes into groups, representing them as fins of the butterfly. We therefore called such a bias model as “butterfly model”. Since each radius represented like a spoke of the wheel, it was also called as “spoke-and-wheel model”. Because there were 13, 9, 2, and 5 AI attributes corresponding to each cluster (or group), so four fins were designed in this butterfly model. It was however important to put these AI studies in a rectangular grid arranged in the order from low-bias to high-bias order. Note that it was therefore imperative to have the product of rows and columns of the grid to be an even number, which represented the total number of studies. In our case, we had 32 studies, the grid consisted of 4×8 (or 8×4) layout, which is shown in Figure 16. Here, the last number is the bias score for that particular study (each bias map has a name as: “Sn-Name:BiasValue”, i.e., “S19-Mes:13”, showing the number of study as 19, and the first three letter of the first author’s last m]name as Mes. The ranking-based on these weights were shown in the (see Table S2, top).

Figure 16.

Radial-bias map for DL-based studies. DL, deep learning.

Radial-bias area method

The main concept of radial-bias area (RBA) method comes from the area occupied by the clusters unlike the spoke-and-wheel model that takes only the consideration of radial strengths. So, RBA consists of finding the “fin strength”, rather than “radial strength”. So, RBA is a cumulative effect of all the components of the cluster. Since area is the basic ingredient of the method, we just need the visual area representation arranged from low area (low-bias) to high area (high-bias), and this is presented in Figure 17 in the form of the same grid pattern as RBM, where the white region represent the bias region. The bias study is represented by the nomenclature: “Sn-Name:BiasValue”, similar to RBM method. A typical example for a non-vascular application is given by “S18-Azz:125”, where “18” designates the study number, “Azz” represents the author’s first three alphabets of the last name, and “125” represent the calculated normalized bias value. Higher bias area is visually reflected by the white zone of the pie chart as shown in Table S2, bottom.

Figure 17.

Radial-bias area for DL-based studies. DL, deep learning.

ROBINS-I

In this method, the bias attempts to simulate randomization in non-randomized trials. The idea in ROBINS-I method is to consider the simulation for randomization in a non-randomized trial framework. For studying the RoB, we adopt three kinds of components namely, (I) “pre-intervention”, (II) “during an intervention”, and (III) “post-intervention”. These three elements are divided into seven separate aspects, including: (I) effect of confounding, leading to bias (total patients, family history, and risk factor), (II) selection of participants, leading to bias (BMI, ethnicity, hypertension, smoking, CUSIP, and organ), (III) types (classification) of interventions, leading to bias (number of algorithms used, DL layers used, and protocol), (IV) deviations from intended interventions, leading to bias (number of PE parameters and sensitivity), (V) role of missing data, leading to bias (handling imbalanced data), (VI) measurement of performance outcomes, leading to bias (accuracy, P value, Dice, and Jaccard), and (VII) selection of the reported result, leading to bias (statistical analysis, validation, stat test, and clinical settings).

A set of 32 studies were taken into consideration for RoB analysis using ROBINS-I. Using the cut-off of 3.7 and 3.1 corresponding to low-bias and moderate-bias, respectively, we found that 25% (8 out of 32), 25% (8 out of 32), and 50% (16 out of 32) were low-bias, moderate-bias, and high-bias, respectively (Figure 18) (Table S3).

Figure 18.

Cumulative plot for all the DL-based CVD studies using ROBINS-I. DL, deep learning; CVD, cardiovascular disease; ROBINS-I, risk of bias in non-randomized studies-of interventions.

PROBAST

It uses predictors classified into four domains, namely, Four kinds of constituents were used for classification of the predictors, namely, (I) participants (total patients, family history, risk factors, BMI, hypertension, smoking, CUSIP, and organ), (II) predictors (number of algorithms used, DL, feature extraction, layers used, and protocol), (III) outcome (sensitivity, accuracy, P value, Dice, and Jaccard), and (IV) analysis (statistical analysis, validation, stat test, and clinical setting).

RoB analysis using the PROBAST tool was performed on a set of 32 studies. Using the cut-off of 3.8 and 3.1 for low-bias and moderate bias, respectively, we found that 21.8% (7 out of 32), 34.4% (11 out of 32), and 43.7% (14 out of 32) were low-bias, moderate-bias, and high-bias, respectively (Figure 19) (Table S4).

Figure 19.

Cumulative plot for all the DL-based CVD studies using PROBAST. DL, deep learning; CVD, cardiovascular disease; PROBAST, prediction model risk of bias assessment tool

Venn diagram for analysing the five bias strategies

The Venn diagram (VD) tool was used to understand the similarities between thee five bias methods namely, RBS, RBM, RBA model, ROBINS-I, and PROBAST. For best representation of bias using VD, we divided the pictorial representation into four categories such as (I) low-bias (Figure 20A), (II) moderate-bias (Figure 20B), (III) high-bias (Figure 20C), and (IV) moderate-high bias (Figure 20D). Our observation showed that the common studies for RBM, RBS, RBA, ROBINS-I, and PROBAST were 4 (40%), 4 (20%), 8 (42.1%), and 21 (75%) in low-bias, moderate-bias, high-bias, and moderate-high-bias, respectively.

Figure 20.

Venn-Diagram (A) low bias; (B) moderate bias; (C) high bias; (D) moderate-high bias. PROBAST, prediction model risk of bias assessment tool; ROBINS-I, risk of bias in non-randomized studies-of interventions.

Critical discussion

Principal findings

Our CVD prediction study primarily focused on DL approaches for preventive-based risk assessment. The architectures for the DL-based systems were discussed along with the unique characteristics of these systems. Thirty-two studies were considered in the DL-based CVD paradigm for statistical analysis. It was observed that the main cause for approaching the DL is the automated feature selections from the input risk factors provided to the system. Our observations clearly showed that DL-based systems have a capability of handling the non-linearity between the covariates (risk factors) and the ground truth (gold standard), unlike the ML-based systems.

The SDL approach was adapted more but with advancement in the AI paradigm, HDL has started to show better results compared to SDL due to the fusion of multiple models. The algorithm used mostly in the DL paradigm was in the following order. For SDL, it was CNN, UNet, RCNN, UNet++, and for HDL, it was SegNet-UNet, CNN-UNet, and ResNet. The RoB analysis was also presented in the DL paradigm for CVD risk stratification by using five different methods (RBS, RBM, RBA, ROBINS-I, and PROBAST). The bias groups were low-bias (6 studies), moderate-bias (11 studies), and high-bias (15 studies) from the RBS model. Also, the factors responsible for the bias were determined. A comparative analysis was also presented and shown in a Venn diagram.

Benchmarking

Table 7 presents the benchmarking table comprising fourteen review studies (9,11,13,80,100-118) for CVD risk stratification using AI (ML or DL). The table has eleven attributes along column 2 to column 12 for every corresponding study from row 2 to row 15. These eleven attributes presented were (I) the studies, (II) the year of the study, (III) the AI specification of the study, (IV) the objective of the study, (V) the usage of the PRISMA model, (VI) the role of statistical classification, (VII) the field of application, (VIII) architectural classifications of the study, (IX) bias analysis was performed or not in the study, (X) number of studies used, and (XI) represents the citations present in the study. The studies which used both ML and DL approaches were row R1 to row R4, and row R7 to row R11, while the studies which implemented the DL approaches were row R5, R6, R12, and R14. The objectives were segmentation of plaque, risk assessment or stratification, detection, and comparison among the ML or DL techniques. The PRISMA model was adopted by six studies. The statistical classification was done by six studies, while others have not shown the classification based on the statistics. The applications were CVD, MI, myocardial perfusion (MP), heart failure, stroke, CKD, and RA. The number of studies that were used for the analysis ranged from 12 to 229, while the total number of citations was from 229 to 40 (286).

Table 7. Benchmarking table.

SN	Studies	Year	AI Spec.	Obb.	PRISMA	Stat. Classn	Application	Arch. Classn	RoB	# of studies	T. citations
1	Jamthikar et al. (13)	2021	ML/DL	Risk assessment	√	√	CVD, CKD	√	×	120	120
2	Biswas et al. (80)	2021	ML/DL	Segmentation	×	√	CVD	√	×	×	163
3	Saba et al. (112)	2021	ML/DL	Comparison	√	√	CVD	√	×	229	229
4	Faizal et al. (114)	2021	ML/DL	Risk prediction	×	×	CVD	√	×	139	139
5	Hinai et al. (115)	2021	DL	Detection	√	√	MI	√	×	12	48
6	Lin et al. (113)	2021	DL	Risk assessment	×	×	CAD	√	×	18	58
7	Yasmin et al. (117)	2021	ML/DL	Detection	×	×	Heart Failure	√	×	22	128
8	Monti et al. (116)	2020	ML/DL	Detection	×	×	CAD, MP	√	×	×	40
9	Jamthikar et al. (11)	2020	ML	Risk assessment	√	√	CVD	√	×	120	120
10	Khanna et al. (110)	2019	ML/DL	Risk assessment	×	×	RA, CVD	√	×	150	150
11	Saba et al. (111)	2019	ML/DL	Risk assessment	√	×	CVD, Stroke	√	×	111	111
12	Krittanawong et al. (118)	2019	DL	Comparison	×	×	CVD	√	×	20	105
13	Banchhor et al. (9)	2018	ML	Stratification	×	×	CVD	√	×	153	153
14	Proposed Study	2022	DL	Stratification	√	√	CVD, COVID	√	√	286	286

SN, serial number; AI Spec., AI specialization; ML, machine learning; DL, deep learning; Obb., objective; Stat. Classn, statistical classification; Arch. Classn, architectural classification; ^#, Number; T. citations, total citations; CVD, cardiovascular diseases; CKD, chronic kidney diseases; MI, myocardial infraction; MP, myocardial perfusion; CAD, coronary artery diseases; RA, rheumatoid arteries.

Cardiovascular risk stratification has evolved further using DL techniques. It has offered the advantage of automated feature extraction from the risk factors such as conventional risk factors, namely, OBBM, LBBM, CUSIP, and MedUSE during the DL training process. The DL techniques were used for the segmentation of CT, US, and MRI images for obtaining the image phenotypes, which were used in combination with OBBM, and LBBM risk factors. The types of DL techniques are (I) solo DL, and (II) hybrid DL. The SDL technique is basically the use of one type of architecture such as CNN, UNet, and UNet++, while the hybrid is the combination of two or more different DL architectures such as SegNet-UNet, etc. The HDL technologies are superior because of their larger scope of feature extraction, classifier diversity that includes softmax or ML-based classifiers, transfer learning features, better performance results, usage of more hardware resources, and diverse application fields as compared to SDL approaches. The HDL has a higher complex program while SDL has low complexity.

Conclusions

The proposed review presented different deep learning-based approaches for CVD risk prediction paradigm, namely solo deep learning, and hybrid deep learning. PRISMA strategy was adapted and the best studies were filtered. Statistical distribution based on the characteristics was carried out along with the detailed architectural classification for the DL-based paradigm. Our observations showed that solo deep learning dominated the CVD industry. The most adopted architecture for CVD risk stratification was CNN followed by UNet, UNet++, and RCNN. Plaque tissue characterization was also conducted for CVD risk stratification. Finally, we conducted bias in AI studies to determine two cut-offs. Five different strategies, namely, RBS, RBM, RBA, ROBINS-I, and PROBAST models for bias estimation. The study showed CVD risk stratification in COVID-19 patients. ECG-based strategies were also presented in this study. As the DL-based approaches are evolving, ensemble-based techniques are likely to emerge for superior performance for CVD risk stratification.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.