Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Mar 23.
Published in final edited form as: JACC Cardiovasc Imaging. 2020 Oct 28;14(3):615–625. doi: 10.1016/j.jcmg.2020.08.024

Machine learning adds to clinical and CAC assessments in predicting 10-year CHD and CVD deaths

Rine Nakanishi a,b,*, Piotr J Slomka c,d,*, Richard Rios c, Julian Betancur c, Michael J Blaha e, Khurram Nasir f, Michael D Miedema g, John A Rumberger h, Heidi Gransar c, Leslee J Shaw i, Alan Rozanski j, Matthew J Budoff b, Daniel S Berman c,d
PMCID: PMC7987201  NIHMSID: NIHMS1631525  PMID: 33129741

Abstract

Objectives:

We aimed to evaluate whether machine learning (ML) of non-contrast CT and clinical variables improve the prediction of atherosclerotic cardiovascular disease (ASCVD) and coronary heart disease (CHD) deaths as compared to coronary artery calcium (CAC) Agatston scoring and clinical data.

Background:

The CAC score provides a measure of the global burden of coronary atherosclerosis, and its long-term prognostic utility has been consistently shown to have incremental value over clinical risk assessment. However, current approaches fail to integrate all available CT and clinical variables for comprehensive risk assessment.

Methods:

The study included data from 66,636 asymptomatic individuals (54±11 years, 67% Male) without established ASCVD undergoing CAC scanning and followed for CVD and CHD deaths at 10 years. Clinical risk assessment employed the ASCVD risk score. For ML we used an ensemble boosting approach to fit a predictive classifier for outcomes followed by automated feature selection using information gain ratio. The model building process used all available clinical and CT data, including the CAC score, the number, volume and density of CAC plaques, and extracoronary scores, comprising a total of 77 variables. We evaluated our overall proposed model (ML all) using a 10-fold cross-validation framework on the population data and area under the curve (AUC) as metrics. The prediction performance was also compared with two traditional scores (ASCVD risk and CAC score) and two additional models that were trained using all the clinical data (ML Clinical) and CT variables (ML CT).

Results:

AUC by ML All (0.845) for predicting CVD death was superior compared to that obtained by clinical data alone (0.821), CAC score alone (0.781) and ML-CT alone (0.804) (p<0.001 for all). Similarly, for predicting CHD death, AUC by ML All (0.860) was superior to the other analyses (0.835 for clinical data, 0.816 for CAC, and 0.827 for ML-CT, p<0.001).

Conclusions:

The comprehensive ML model was superior to clinical risk factors, CAC scores, and a ML model fitted using CT variables alone in prediction of both CVD and CHD deaths.

Keywords: Machine learning, Coronary artery calcification, Coronary heart disease death, Cardiovascular disease death, Pooled cohort equation

Condensed Abstract:

The study included data from 66,636 asymptomatic individuals without established atherosclerotic cardiovascular disease (ASCVD) undergoing coronary artery calcium (CAC) scanning and followed for cardiovascular disease (CVD) and coronary heart disease (CHD) deaths at 10 years. Comprehensive machine learning (ML) used 77 clinical and CT variables, including: the number, volume and density of CAC plaques, CAC and extracoronary scores, among others. Risk estimation by ML was superior to the current traditional risk equation and CAC score for prediction of CVD and CHD deaths and demonstrated high concordance between ML-predicted and actual observed risk for both CVD and CHD deaths.

Introduction

In clinical practice traditional risk factors obtained from population-based studies are used to predict cardiovascular disease (CVD) events. For example, the 2013 ACC/AHA atherosclerotic cardiovascular disease (ASCVD) risk estimator has improved risk assessment of cardiovascular diseases compared to previous risk algorithms 1. However, current clinical models still misclassify future risk assessment (14). Several studies, for instance, have shown that the ASCVD risk estimator and all other current risk scores overestimate actual observed risk (14).

Coronary artery calcification is a robust marker of coronary atherosclerosis. The coronary artery calcium (CAC) score measured by non-contrast cardiac-gated computed tomography (CT) provides a measure of the global burden of coronary atherosclerosis, reflecting the effect all measured and unmeasured risk factors causing coronary atherosclerosis in an individual patient. Its long-term prognostic value has been shown consistently to provide independent predictive information to assess clinical risk of CVD and coronary heart disease (CHD) events (59). In addition, other CT variables such as the total number of calcified coronary lesions, plaque density and thoracic aorta calcification have been demonstrated to add to CAC assessment in prediction of CVD events (10,11). However, current models fail to integrate all available CT and clinical variables for comprehensive risk assessment.

Machine learning (ML) is the scientific field that enables data-driven predictions by learning from data. ML builds models that can learn from training samples to subsequently perform prediction tasks in unseen samples. ML techniques have showed equal or better performance than humans in medical tasks such as diagnosis, decision-making and risk prediction in cardiology (1216). This is the first study, to our knowledge, to assess the prognostic value of ML to estimate CVD and coronary heart disease (CHD) deaths among asymptomatic individuals integrating clinical and CAC data. We hypothesized that comprehensive ML of CAC and other variables from non-contrast cardiac CT can better predict CHD and CVD deaths than current state-of-the art methods for risk prediction. The aim of the current study was then to evaluate whether ML, considering all available clinical and cardiac CT imaging variables, predicts CVD and CHD deaths more accurately than existing assessments.

Methods

Study population

The CAC Consortium is a large multicenter observational cohort study of patients who have undergone CAC scanning for clinical purposes and is designed to determine the cause-specific death including CVD, CHD and non-CVD deaths.

Details regarding the CAC consortium have been described previously (17). The CAC consortium includes 66,636 asymptomatic individuals (54±11 years, 67% Male) without known CHD who non-contrast cardiac CT for detecting CAC at 4 high volume centers in the U.S (Harbor UCLA Medical Center, Torrance, California; Cedars Sinai Medical Center, Los Angeles, California; Columbus, Ohio; and Minneapolis Heart Institute, Minneapolis, Minnesota). All sites had at least 10 years’ experience to exam CAC scanning, provided >5000 scans per site and can complete >90% of clinical demographics which were required for the study. Inclusion criteria were patients with ≥18 years old, asymptomatic, no history of CHD, and who underwent CAC scanning. Exclusion criteria were missing data of scan identifiers (n=2650), no-dedicated CAC score (n=4669), no-CAC scanning (n=4833), uncertain date of birth (n=150), uncertain data of scan (n=11), or insufficient data for follow-up (n=10,320).

Each institution obtained Institutional Review Board approval and all participants provided informed consent.

Clinical demographics

Clinical demographics and laboratory data were collected at the time of CAC scanning or at a clinical visit associated with the scan. Hypertension, diabetes and dyslipidemia were defined when individuals self-reported diagnosis made by their physicians, had testing at the time of the scan visit, or had been treated by medications for these diseases. Dyslipidemia was also defined when LDL-C >160 mg/dL, HDL-C<40 mg/dL in men and <50 mg/dL in women, or fasting triglycerides>150 mg/dL were present. Never, former, or current smoking was recorded for smoking status. Family history of CHD was defined as premature family history (<55 years in old in a male relative and <65 years old in a female relative) at the Columbus, Ohio site, or the presence of a first-degree relative with a history of premature CHD at other 3 sites. The ASCVD risk score was calculated by using the PCE (17). Total cholesterol and high-density lipoprotein (220 mg/dL and 40mg/dL for patients with untreated dyslipidemia; 190mg/dL and 60mg/dL for patients without dyslipidemia; 180mg/dL and 50mg/dL for dyslipidemic patients with treatment) were used to calculate ASCVD risk score. When historical risk-factors did not include measurements regarding blood pressure, 150mmHg and 90mmHg were used as systolic and diastolic blood pressure for hypertensive patients without treatment. 135 mmHg and 85mmHg for hypertensive patients with treatment, and 120mmH and 80mmHg for patients without hypertension were used for calculating ASCVD risk score.

Study Follow-up

Death was defined by patient identifiers including social security number, name, date of birth through the Social Security Death Index (SSDI) Death Master File and followed through June 1st 2014. Cause of death was determined by coded death certificates through the National Death Index service. The maximum follow-up time was truncated at 10 years to investigate actual 10 years CVD/CHD risk prediction by ML.

CT protocol and interpretation of CAC

CAC scans were performed in accordance with standard protocols (18). Because of the current study nature to investigate >10-year death for patients, electron beam tomography was obtained in approximately 93% of patients. In total, approximately 13%, 38%, 38% and 3.5% of patients were scanned with the Imatron C-100 scanner, the C-150, the C-300, and the e-Speed scanner (GE-Imatron), respectively. More recent data at two sites was collected using multidetector CT in 7% of patients on a 4-slice MDCT scanner (Somatom Volume Zoom, Siemens Medical Solutions) and the General Electric LightSpeed VCT 64-slice platform (GE Healthcare). Due to the long-term follow-up, most of the scans (>90%) were performed by electron beam tomography and the rest were scanned by multidetector CT. CAC and extracoronary calcification including thoracic aortic calcification (TAC), aortic valve calcification (AVC) and mitral valve calcification (MVC) were scored using Agatston method (19). Besides, CAC scores as well as volume score and mean CAC densities for left main and other main three vessels were also available. Additional information regarding CAC including total number of CAC plaques, CAC volume scores, CAC density, TAC scores, AVC scores and MVC scores were available in 68%, 51%, 30%, 51%, 15% and 15% of the cohort, respectively.

Machine learning

Figure 1 illustrates the steps followed to train and evaluate the proposed model (ML all) using a 10-fold cross validation framework in the study population. First, the overall population was randomly divided into 10 equally sized non-overlapping groups. One group containing 10% was retained as the test set and the other 90% were used as the training set; second, a feature selection was performed using the training set and information gain; third, a data-driven model was fitted using the training set and an ensemble boosting approach (LogitBoost); fourth, the prediction performance was evaluated using the test set. The cross-validation procedure then looped 10 times over the various groups, each time performing variable selection and model building, and using different training and test sets - meaning that none of the data points were used for model training and evaluation at the same time. We used this validation procedure seeking to maximize the use of training and validation data, avoid the testing of hypothesis suggested by arbitrary splitting of data, and reduce the variance in prediction error. Once finished, as fifthly step, the predictions of the corresponding 10 models were stacked to assess the overall prediction performance of CHD and CVD deaths.

Figure 1. Workflow of the method used to train the proposed model.

Figure 1.

1) Random split of population in 10 folds, 2) Selection of variables using information gain, 3) k-th model building using ensemble boosting, 4) Evaluation of prediction performance of k-th model, 5) evaluation of overall prediction of CHD and CVD events.

Variable selection

A total of 77 variables, including 46 clinical variables (e.g. ASCVD risk score, age, sex, race, body mass index, hypertension, diabetes, hyperlipidemia, current smoking, family history of CHD, smoking years, and medication information) and 31 CT variables (derived from CAC scans) (Table 2), were available to train the model. We firstly used information gain to select the best attributes for the classifier using those variables that resulted in an information gain > 1e-5. Information gain is a measure of the amount of information gained from the data by attribute (20,21).

Table 2.

Clinical and CT variables

Clinical variables CT variables
CAC variables Non-CAC variables
ASCVD risk score Diabetes Total CAC score Presence of TAC
Age Oral diabetic medications LM CAC score TAC score
Sex Insulin LAD CAC score TAC volume score
Race Glucose LCx CAC score Presence of AVC
Height Current smoker RCA CAC score AVC score
Weight Past smoker Total volume score AVC volume score
Obese Smoking years LM volume score Presence of MVC
Body mass index Smoking pack years LAD volume score MVC score
Heart rate Smoking packs LCx volume score MVC volume score
Menopause Digoxin RCA volume score Descending aorta diameter
Site Statins Total CAC lesions Ascending aorta diameter
Fasting Nitrates LM CAC lesions
Hypertension Angiotensin-converting enzyme (ACE) inhibitors LAD CAC lesions
Hypertension medications Beta blockers LCx CAC lesions
Systolic blood pressure Vitamin C RCA CAC lesions
Diastolic blood pressure Aspirin Total CAC mean density
Dyslipidemia Niacin LM CAC mean density
Dyslipidemia medications Calcium blockers LAD CAC mean density
Low density lipoprotein Blood thinner medications LCx CAC mean density
Cholesterol Stroke RCA CAC mean density
High density lipoprotein Peripheral vascular disease
Triglyceride Kidney disease
Lung disease
Family history

Abbreviations: CT-Computed tomography, ASCVD-Atherosclerotic cardiovascular disease, CAC-Coronary artery calcium, LM-Left main, LAD-Left anterior descending artery, LCx-Left circumflex, RCA-Right coronary artery, TAC-Total aorta calcium, AVC-Aortic valve calcium, MVC-Mitral valve calcium

Model building

We used an ensemble boosting approach to fit a predictive classifier for cardiovascular outcomes (22). This boosting method called LogitBoost is tree-based learning technique that combines the predictions of many weak classifiers to produce a single powerful prediction: A weak learner is fit in each iteration seeking to reduce the misclassification error of previous iterations. For a given patient, the outcome of ML model - called the ML score - was then the probability risk of having CVD and CHD mortality. It is also worth adding that this technique is suitable to deal with missing data (21). This technique uses non-missing data to establish a ranking of surrogate variables: The first surrogate is the feature that best describe the training data while the second surrogate does the second-best description, and so on. It then imputes missing data, either in the training or test phase, using the ranking of surrogate variables in order, if the first surrogate variable is missing. ML and feature selection were implemented in the open-source Waikato Environment for Knowledge Analysis (WEKA) platform 3.8.0 (University of Waikato, Hamilton, New Zealand).

Prediction models

We trained two additional ML models to compare to our proposed model (ML All) following the steps previously described: 1) a first model trained with all the clinical variables (ML Clinical); and 2) a second model trained with all CT variables (ML CT). These two ML models were trained and evaluated using the same folds and cross-validation procedure followed for the ML All model to subsequently enable paired comparisons. We also compared our ML ALL model with a logistic regression model (LR-3F) trained with age, ASCVD and CAC scores to predict both CHD and CVD deaths. This analysis was done to determine the benefit of combining all variables using ML technique such as LogitBoost.

Statistical analyses

We compared the prediction performance of our ML model with the ASCVD risk and CAC score: using AUC as metric to evaluate the overall performance of the ML models and traditional scores. Youden index was also provided to summarize performance predictions. Pairwise comparisons were performed between the ASCVD risk score, CAC score alone, and ML models using DeLong test (23). The ML models were assessed by sex and the Brier scores were computed between predicted and observed CVD and CHD deaths (24). Additionally, ML All model was compared with traditional scores in single random stratified partitioning of our population data. Statistical calculations were performed in R software version 3.4 using the pROC package for DeLong analysis (25).

Results

Table 1 summarizes clinical characteristics of patients in the current study. Mean age was 54±11 years and 67% were males. Most of the study cohort were white. The mean 10-year ASCVD risk score was 7.4±8.9%. Hyperlipidemia and family history of CHD were the most common cardiovascular risk factors, following hypertension, current smoking and diabetes, respectively.

Table 1.

Patient characteristics (n= 66,636)

Age (years) 54±11
Male (n, %) 44,633 (67)
White/Black/Hispanic/Others (%) 89/2/3/6
BMI (kg/m2) 27.5±5.3
Hypertension (n, %) 20,624 (31)
Diabetes (n, %) 4,503 (7)
Hyperlipidemia (n, %) 36,227 (54)
Current smoking (n, %) 6,400 (10)
Family history of CHD (n, %) 30,720 (46)
ASCVD risk score (mean ± SD) 7.4 ± 8.9%
CAC score (n, %) (n=66,636)
 CAC 0 29,757 (45)
 CAC 1–99 20,534 (30)
 CAC 100–399 7,341 (14)
 CAC ≥400 9,004 (11)
TAC score (n, %) (n=41,066)
 TAC 0 19,476 (48)
 TAC 1–99 11,927 (29)
 TAC 100–399 5,415 (13)
 TAC ≥400 4,248 (10)
AVC score (n, %) (n=10,007)
 AVC 0 8,610 (86)
 AVC 1–99 876 (9)
 AVC 100–399 352 (3)
 AVC ≥400 169 (2)
MVC score (n, %) (n=10,008)
 MVC 0 9,416 (94)
 MVC 1–99 283 (3)
 MVC 100–399 150 (1)
 MVC ≥400 159 (2)

Abbreviations: BMI- Body mass index, CHD- Coronary heart disease, ASCVD- Atherosclerotic cardiovascular disease, CAC- Coronary artery calcium, TAC- Thoracic aortic calcification, AVC- Aortic valve calcification, MVC- Mitral valve calcification

The endpoints of the current study are CHD death (n=524), and CVD death (n=971) including death from CHD (n=524, 54%), stroke (n=160, 17%), congestive heart failure (n=51, 5%), and other circulatory disease (n=236, 24%). The variable rankings (first 50 variables) for prediction of CVD and CHD deaths are listed in Figure 2 (Figure 2a: CVD death, and Figure 2b: CHD death). ASCVD risk score was the feature that obtained the highest gain for predicting both types of deaths, followed by age and CAC score. Similarly, these variables were the most used to train the weak classifiers in the ML All model for predicting both types of risk.

Figure 2a.

Figure 2a.

Importance ranking of variables for prediction of CVD death.

Figure 2b.

Figure 2b.

Importance ranking of variables for prediction of CHD death.

AUCs for predicting the 10-year CVD and CHD deaths are shown in Central Illustration (a and b respectively). For CVD death, AUC for ML Clinical was significantly higher than ML for ASCVD risk alone (AUC; 0.826 vs. 0.821, p<0.001) and ML CT significantly improved the prediction compared to CAC alone (0.804 vs. 0.781, p<0.001). In addition, AUC for ML All (0.845) was higher than that for ML clinical (0.826, p<0.001) and ML CT (0.804, p<0.001) (Central Illustration-a). With respect to CHD death, AUCs for ML Clinical and ASCVD risk alone were comparable (0.835 vs. 0.834, p=0.639). ML CT significantly improved the prediction compared to CAC alone (0.827 vs. 0.816, p=0.025). AUC for ML All (0.860) was higher than that for ML Clinical (p<0.001) or ML CT (p<0.001) (Central Illustration-b). The separate results in male and female cohorts are provided in Supplemental Figure 1. Tables 3 and 4 provide the Youden index for each model and traditional scores, where our ML All model obtained the highest Youden index for both CVH and CHD deaths.

Central illustration-a.

Central illustration-a.

Receiver operating characteristic curves for prediction of CVD death

Central illustration-b.

Central illustration-b.

Receiver operating characteristic curves for prediction of CHD death

Table 3.

Optimal cut-points and Youden index for each model in 10-year CHD deaths

Model Optimal cut-point Sensitivity Specificity Youden index
ML All 0.005 0.825 0.745 0.570
ML Clinical 0.007 0.720 0.802 0.522
ASCVD risk 0.114 0.746 0.783 0.529
ML CT 0.007 0.748 0.778 0.526
CAC score 76.80 0.781 0.732 0.512

Table 4.

Optimal cut-points and Youden index for each model in 10-year CVD deaths

Model Optimal cut-point Sensitivity Specificity Youden index
ML All 0.011 0.737 0.803 0.540
ML Clinical 0.010 0.742 0.761 0.504
ASCVD risk 0.113 0.720 0.780 0.500
ML CT 0.013 0.731 0.765 0.496
CAC score 56.0 0.751 0.702 0.453

A comparative plot of observed and ML predicted risks is shown in Figure 3. An excellent Brier score (<0.05) was shown for prediction of CVD and CHD deaths, resulting in precise ML-prediction of CVD (Figure 3a) and CHD death (Figure 3b) compared to observed deaths.

Figure 3.

Figure 3.

A comparative plot of observed and ML predicted risks for cardiovascular (A) and cardiac (B) death.

In the sub-analyses to investigate if ML ALL improved the prognostic predictions compared to conventional analysis, ML ALL improved the prognostic prediction of a logistic regression model trained with age, ASCVD and CAC scores (LR-3F) (Supplemental Figure 2).

When the ML All model was compared with traditional scores in single random stratified partitioning of the population data into training (80%) and test (20%) sets, we obtained the same tendency of variable rankings and performance predictions for both CVD and CHD death (Supplemental Figure 3) and the AUC for ML was higher than that for ASCVD risk and CAC score (Supplemental Figure 4).

Discussion

In a large multicenter cohort of 66,636 asymptomatic individuals undergoing clinical CAC scans, we have showed that ML models that integrates all clinical and non-contrast CT imaging variables can obtain superior prognostic performance than traditional scores to predict both CVD and CHD deaths. In this study, we developed a ML method to show the added prognostic value of integrating clinical and non-contrast CT imaging variables, providing a ranking of the most significant variables to predict CVD and CHD deaths. Our comprehensive ML model obtained high concordance between the ML risk score and actual observed risk for both CVD and CHD deaths, suggesting prospective clinical implementations of ML models to assess the risk for both types of deaths. We have demonstrated that age, ASCVD risk and CAC score showed highest gains to predict events and that additional variables provide significant, incremental value in the overall prediction performance.

The standard approach to analyze the predictive value of CAC scanning along with clinical information has been performed on a limited number of variables. The ASCVD risk score, for example, is a parsimonious score based on a handful of these available clinical variables. Also, prognosis with CAC scanning is typically assessed alone without consideration of variables such as CAC score in individual vessels, number of plaques, or extracoronary calcification. While a comprehensive method for integrating all available variables from the CAC scan has been advocated (26), such a method has not been developed. Further, a comprehensive integration of all available clinical information with a more complete analysis of variables from non-contrast CT has also not been proposed. In the current study, we developed a ML method that integrating clinical and non-contrast cardiac CT imaging data outperformed the traditional scores and single-variable-type ML models on the prognostic prediction for CVD and CHD death: Similarly by gender, we showed that our ML method outperformed the prognostic prediction in females and males for CVD death. In the current study the prognostic improvement of the ML method was significant for CHD death in males but not in females. The reason for this finding is not clear. A prior study from the CAC Consortium revealed that after adjustment for ASCVD risk, CAC scores added significantly prognostic accuracy in both males and females for prediction of CVD death (reports for CHD death were not made) (27).

Conventional risk assessment involves classification of patients into few predefined risk categories. Without precise quantitative estimates, however, crude risk categorization with arbitrary thresholds may misclassify patients compared to continuous risk predictions (28). In contrast, ML can give a precise risk calibration for specific patient. Recent studies have motivated the development of comprehensive risk assessment by ML with imaging based on coronary CT angiography or myocardial single photon emission CT (29,30). In this study, we used a similar method to maximize non-contrast CT information for cardiovascular risk prediction in a large asymptomatic population. Our findings suggest that future risk prediction models based on all available information can achieve a more accurate and precise model to identify risk that could be implemented clinically to improve the clinical use of CAC scanning in risk assessment and guiding management decisions (14,15,30). The ML approach is likely to become a routine tool for risk assessment using CAC scanning with the evolution of the electronic medical record and its integration with imaging data in future clinical practice.

Limitations

Our study has some limitations. Although the available CT data contained several variables in addition to the CAC score, multiple CT variables were not available in all patients, introducing uncertainties that may affect the prediction performance of the ML model. Additional variables of prognostic importance such as epicardial adipose tissue were not contained in the database. Our outcome variables were CVD and CHD deaths, since the CAC Consortium database did not contain regarding nonfatal cardiac events. CHD and CVD deaths overlap, with the CHD deaths being included in the latter. By including the latter, we are able to show that CAC is as predictive for both CVD and CHD events. There were a relatively small number of CHD deaths in females. This may possibly explain that the improvement did not reach statistical significance in AUC of ML All over ML Clinical for CHD death in females. We used CVD and CHD deaths for the primary outcomes in the current study since the CAC Consortium was deigned to determine cause-specific death including CVD, CHD and non-CVD deaths among asymptomatic patients undergoing CAC.

Conclusion

The current study demonstrated that a ML approaches that integrate clinical and non-contrast CT variables can provide better risk assessment of CVD and CHD death than the combination of the traditional ASCVD and CAC scores. As clinical data from digital medical records become available for seamless integration with imaging data, the ML approach is likely to become a routine tool for clinical risk assessment using CAC scanning in the future.

Supplementary Material

Supplementary Material

Clinical Perspectives:

COMPETENCY IN MEDICAL KNOWLEDGE:

The integration of clinical and non-contrast CT imaging variables into ML methods enables to develop risk models that have superior prognostic prediction of CVD and CHD death than the traditional clinical risk scores such as ASCVD and CAC scores.

TRANSLATIONAL OUTLOOK:

Breakthroughs in electronic medical records and integration systems with imaging workstations can facilitate the implementation of ML models as a routine tool for clinical risk assessment using clinical and CAC scanning in the future.

Acknowledgments

Financial Support: This research was supported in part by grant R01HL089765 from the National Heart, Lung, and Blood Institute/ National Institutes of Health (NHLBI/NIH) (PI: Piotr Slomka). Dr. Blaha has received support from NIH award for this project (L30 HL 110027). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. At Cedars-Sinai, the study was supported in part by a grant from the Miriam & Sheldon G. Adelson Medical Research Foundation (PI: Dr Berman). Dr. Budoff has served as a consultant for General Electric.

Abbreviations

CAC

Coronary artery calcium score

ML

Machine learning

ASCVD

Atherosclerotic cardiovascular disease

CVD

Cardiovascular disease

CHD

Coronary heart disease

PCE

Pooled cohort equation

LR

Logistic regression

Footnotes

The other authors have no conflict of Interest.

References

  • 1.Kavousi M, Leening MJ, Nanchen D et al. Comparison of application of the ACC/AHA guidelines, Adult Treatment Panel III guidelines, and European Society of Cardiology guidelines for cardiovascular disease prevention in a European cohort. Jama 2014;311:1416–23. [DOI] [PubMed] [Google Scholar]
  • 2.DeFilippis AP, Young R, Carrubba CJ et al. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med 2015;162:266–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.DeFilippis AP, Young R, McEvoy JW et al. Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort. Eur Heart J 2017;38:598–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Rana JS, Tabada GH, Solomon MD et al. Accuracy of the Atherosclerotic Cardiovascular Risk Equation in a Large Contemporary, Multiethnic Population. J Am Coll Cardiol 2016;67:2118–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Budoff MJ, Shaw LJ, Liu ST et al. Long-term prognosis associated with coronary calcification: observations from a registry of 25,253 patients. J Am Coll Cardiol 2007;49:1860–70. [DOI] [PubMed] [Google Scholar]
  • 6.Nakanishi R, Li D, Blaha MJ et al. All-cause mortality by age and gender based on coronary artery calcium scores. Eur Heart J Cardiovasc Imaging 2016;17:1305–1314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Nakanishi R, Li D, Blaha MJ et al. The relationship between coronary artery calcium score and the long-term mortality among patients with minimal or absent coronary artery risk factors. Int J Cardiol 2015;185:275–81. [DOI] [PubMed] [Google Scholar]
  • 8.McClelland RL, Jorgensen NW, Budoff M et al. 10-Year Coronary Heart Disease Risk Prediction Using Coronary Artery Calcium and Traditional Risk Factors: Derivation in the MESA (Multi-Ethnic Study of Atherosclerosis) With Validation in the HNR (Heinz Nixdorf Recall) Study and the DHS (Dallas Heart Study). J Am Coll Cardiol 2015;66:1643–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Erbel R, Möhlenkamp S, Moebus S et al. Coronary risk stratification, discrimination, and reclassification improvement based on quantification of subclinical coronary atherosclerosis: the Heinz Nixdorf Recall study. J Am Coll Cardiol 2010;56:1397–406. [DOI] [PubMed] [Google Scholar]
  • 10.Blaha MJ, Budoff MJ, Tota-Maharaj R et al. Improving the CAC Score by Addition of Regional Measures of Calcium Distribution: Multi-Ethnic Study of Atherosclerosis. JACC Cardiovasc Imaging 2016. [DOI] [PMC free article] [PubMed]
  • 11.Mahabadi AA, Lehmann N, Mohlenkamp S et al. Noncoronary Measures Enhance the Predictive Value of Cardiac CT Above Traditional Risk Factors and CAC Score in the General Population. JACC Cardiovasc Imaging 2016;9:1177–1185. [DOI] [PubMed] [Google Scholar]
  • 12.Siegersma KR, Leiner T, Chew DP, Appelman Y, Hofstra L, Verjans JW. Artificial intelligence in cardiovascular imaging: state of the art and implications for the imaging cardiologist. Neth Heart J 2019;27:403–413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Deo RC. Machine Learning in Medicine. Circulation 2015;132:1920–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Arsanjani R, Xu Y, Dey D et al. Improved accuracy of myocardial perfusion SPECT for detection of coronary artery disease by machine learning in a large population. J Nucl Cardiol 2013;20:553–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Arsanjani R, Dey D, Khachatryan T et al. Prediction of revascularization after myocardial perfusion SPECT by machine learning in a large population. J Nucl Cardiol 2015;22:877–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Narula S, Shameer K, Salem Omar AM, Dudley JT, Sengupta PP. Machine-Learning Algorithms to Automate Morphological and Functional Assessments in 2D Echocardiography. J Am Coll Cardiol 2016;68:2287–2295. [DOI] [PubMed] [Google Scholar]
  • 17.Blaha MJ, Whelton SP, Al Rifai M et al. Rationale and design of the coronary artery calcium consortium: A multicenter cohort study. J Cardiovasc Comput Tomogr 2017;11:54–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Abbara S, Blanke P, Maroules CD et al. SCCT guidelines for the performance and acquisition of coronary computed tomographic angiography: A report of the society of Cardiovascular Computed Tomography Guidelines Committee: Endorsed by the North American Society for Cardiovascular Imaging (NASCI). J Cardiovasc Comput Tomogr 2016;10:435–449. [DOI] [PubMed] [Google Scholar]
  • 19.Agatston AS, Janowitz WR, Hildner FJ, Zusmer NR, Viamonte M Jr., Detrano. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol 1990;15:827–32. [DOI] [PubMed] [Google Scholar]
  • 20.Quinlan JR. Induction of Decision Trees, 1986.
  • 21.Daniel Berrar WD. Information gain (KullbackLeibler divergence). In: Dubitzky W, Wolkenhauer O, Cho KH, Yokota H (eds) Encyclopedia of Systems Biology Springer, New York, NY: 2013:1022–1023. [Google Scholar]
  • 22.Jerome Friedman TH, Tibshirani Robert. Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors). Ann Statist 2000;28:337–4–7. [Google Scholar]
  • 23.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [PubMed] [Google Scholar]
  • 24.BRIER GW. Verification of forecasts expressed in terms of probability. Mon. Wea. Rev, 1950:1–3.
  • 25.Team RC. R: A Language and Environment for Statistical Computing. : Open Journal of Statistics, 2017.
  • 26.Berman DS, Arnson Y, Rozanski A. Assessment of Coronary Calcium Density on Noncontrast Computed Tomography. JACC Cardiovasc Imaging 2017;10:855–857. [DOI] [PubMed] [Google Scholar]
  • 27.Shaw LJ, Min JK, Nasir K et al. Sex differences in calcified plaque and long-term cardiovascular mortality: observations from the CAC Consortium. Eur Heart J 2018;39:3727–3735. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Wynants L, van Smeden M, McLernon DJ, Timmerman D, Steyerberg EW, Van Calster B. Three myths about risk thresholds for prediction models. BMC medicine 2019;17:192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Betancur J, Rubeaux M, Fuchs TA et al. Automatic Valve Plane Localization in Myocardial Perfusion SPECT/CT by Machine Learning: Anatomic and Clinical Validation. J Nucl Med 2017;58:961–967. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Motwani M, Dey D, Berman DS et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J 2017;38:500–507. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material

RESOURCES