Prediction of Coronary Artery Calcium and Cardiovascular Risk on Chest Radiographs Using Deep Learning

Abstract

Purpose

To assess the ability of deep convolutional neural networks (DCNNs) to predict coronary artery calcium (CAC) and cardiovascular risk on chest radiographs.

Materials and Methods

In this retrospective study, 1689 radiographs in patients who underwent cardiac CT and chest radiography within the same year, between 2013 and 2018, were included (mean age, 56 years ± 11 [standard deviation]; 969 radiographs in women). Agatston scores were used as ground truth labels for DCNN training on radiographs. DCNNs were trained for binary classification of (a) nonzero or zero total calcium scores, (b) presence or absence of calcium in each coronary artery, and (c) total calcium scores above or below varying thresholds. Results from classification of test images were compared with established 10-year atherosclerotic cardiovascular disease (ASCVD) risk scores in each cohort. Classifier performance was measured using area under the receiver operating characteristic curve (AUC) with attention maps to highlight areas of decision-making.

Results

Binary classification between zero and nonzero total calcium scores reached an AUC of 0.73 on frontal radiographs, with similar performance on laterals (AUC, 0.70; P = .56). Performance was similar for binary classification of absolute total calcium score above or below 100 (AUC, 0.74). Frontal radiographs that tested positive for a predicted nonzero CAC score correlated with a higher 10-year ASCVD risk of 17.2% ± 10.9 compared with 11.9% ± 10.2 for a negative test, indicating predicted CAC score of zero (P < .001). Multivariate logistic regression demonstrated the algorithm could predict a nonzero calcium score independent of traditional cardiovascular risk factors. Performance was reduced for individual coronary arteries. Heat maps primarily localized to the cardiac silhouette and occasionally other cardiovascular findings.

Conclusion

DCNNs trained on chest radiographs had modest accuracy for predicting the presence of CAC correlating with cardiovascular risk.

Keywords: Coronary Arteries, Cardiac, Calcifications/Calculi, Neural Networks

See also the commentary by Gupta and Blankstein in this issue.

©RSNA, 2021

Summary

Deep learning algorithms can predict the presence of coronary artery calcium with modest performance on chest radiographs, which is correlated with cardiovascular risk.

Key Points

• ■ Deep convolutional neural networks trained with scores obtained from calcium score CT scans can predict the likelihood that coronary artery calcium (CAC) is present on chest radiographs with modest performance (area under the receiver operating characteristic curve, 0.73).

• ■ Positive prediction of the presence of CAC using machine learning on radiographs is correlated with a higher 10-year atherosclerotic cardiovascular disease risk score (17.2% ± 10.9 vs 11.9% ± 10.2; P < .001).

• ■ Logistic regression analysis demonstrates that the predictive value of the deep learning algorithm is independent of traditional cardiovascular risk factors, showing promise for opportunistic screening of at-risk populations.

Introduction

The Agatston calcium score quantifies the severity of atherosclerotic coronary artery calcium (CAC) and has shown value in predicting future cardiovascular events and all-cause mortality in patients with an intermediate risk for coronary artery disease (1). CAC scores can be used in conjunction with data from the Multi-Ethnic Study of Atherosclerosis (MESA) to offer CAC percentile relative to patients with the same age, sex, and race who are free of clinical cardiovascular disease (2,3). Complementary models, such as the atherosclerotic cardiovascular disease (ASCVD) model, predict the 10-year cardiovascular risk using demographic and laboratory information without imaging data (4).

CAC levels are typically measured at non–contrast material–enhanced electrocardiographically gated cardiac CT and are calculated using the area of calcifications on cross-sectional sections weighted by CT attenuation. CAC remains readily visible on chest CT scans obtained without electrocardiography gating (5), though the prognostic value of coronary artery calcifications was initially demonstrated at non–contrast-enhanced cardiac fluoroscopy and radiography (6,7), predating the development of the Agatston scoring at CT (8). The evaluation and reporting of incidental CAC is accordingly recommended, particularly in patients without suspected coronary artery disease (9).

Chest radiographs are much more commonly obtained than CT scans, though coronary artery calcifications are often overlooked in routine clinical practice. Recent advancements in machine learning have demonstrated the ability of deep convolutional neural networks (DCNNs) to quantify information beyond human perception. This was perhaps most apparent in the 2017 Radiological Society of North America (RSNA) Bone Age Challenge, which illustrated the ability of deep learning to estimate numeric bone age on pediatric radiographs with accuracy to 4.2 months (10). Likewise, DCNNs may be able to accurately assess for coronary artery calcifications and predict cardiovascular risk on chest radiographs.

The purpose of this study was to assess the ability of DCNNs to predict the presence of CAC from chest radiographs and correlate results with established cardiovascular risk assessment models.

Materials and Methods

This study was approved for an exemption by the institutional review board, and the study protocol was compliant with the Health Insurance Portability and Accountability Act.

Study Population

We retrospectively searched for radiographs in adult patients who had undergone both a calcium score CT and posteroanterior and lateral chest radiography within a 12-month period at our tertiary care hospital over a 5-year search period from 2013 to 2018 (Table 1). Calcium-scoring CT scans were obtained for purposes of cardiovascular risk stratification, and chest radiographs were obtained for various indications. The most common chest radiograph indications included chest pain (40.1%), cough (14.2%), shortness of breath (12.8%), and preoperative evaluation (4.0%). If a patient underwent multiple radiography or CT examinations, all were included in the data set, with radiographs linked to the temporally closest calcium scoring CT. Radiograph quality was manually reviewed by a 4th-year radiology resident (P.I.K.). Radiographs that were mislabeled and lateral radiographs where the cardiac silhouette was off the field of view were excluded.

Table 1:

Demographic, Laboratory, and Cardiovascular Risk Characteristics of Patients in the Study

Data Set Curation and Image Preprocessing

A total of 1689 radiographs (mean age of the patient at the time of radiograph, 56 years ± 11; 969 radiographs in women) from approximately 1010 unique patients met the inclusion criteria. Images were anonymized using an in-house anonymization software and converted from Digital Imaging and Communications in Medicine (DICOM) files to JPEG images using the Pydicom (https://pydicom.github.io/) and OpenCV 4.20 (https://opencv.org/) Python libraries. During conversion, images were normalized to 256 grayscale pixel intensity values and rescaled using OpenCV bilinear interpolation to achieve a final resolution of 224 × 224 pixels from the native high-resolution DICOM images.

Calcium score CT examinations were performed with electrocardiography gating across our imaging enterprise, predominantly using dual-source CT scanners (Somatom Definition Flash and Somatom Force; Siemens Healthineers) with the following standard parameters: tube potential of 120 kV, 80 reference mAs, 3-mm section thickness, and pitch of 3.0–3.5. After each examination, CAC results were documented by cardiothoracic radiologists at our institution using structured templates. Reports were parsed using regular expression-based text matching to extract the total Agatston calcium score and the scores for each coronary artery (ie, left main, left anterior descending, left circumflex, and right coronary arteries) (Fig 1). The total and per-vessel Agatston scores were considered ground truth labels for training, validation, and testing of the neural network on radiographs.

Figure 1:

Images in a 65-year-old woman with a total Agatston calcium score of 6056, with scores of 3387 in the right coronary artery and 1304 in the left anterior descending artery. Posteroanterior and lateral chest radiographs demonstrate very faint parallel curvilinear calcifications along the, A and B, right coronary artery and, D and E, left anterior descending artery distribution, which are much more apparent on, C and F, corresponding chest CT images.

The following relevant demographic and clinical information was obtained: age, sex, ethnic and racial identification, history of smoking, diabetes, hypertension, hyperlipidemia, stroke, chronic kidney disease, and numeric values for total cholesterol level, high-density lipoprotein level, systolic blood pressure, and diastolic blood pressure. Ten-year ASCVD risk scores for patients with available data were calculated using publicly available Python libraries (4,11). Age was determined from the time the radiograph was obtained. If there were multiple radiographs for a patient in the data set, laboratory values and demographic information that were temporally closest to the radiograph were linked to the radiograph. The MESA online calculator (https://www.mesa-nhlbi.org/Calcium/input.aspx) was used to compute each patient's CAC percentile relative to others with the same age, sex, and race or ethnicity (12).

DCNN Development

Radiographs were assigned into approximately 70% training, 15% validation, and 15% holdout testing sets, assuring approximately equal distribution of classes in each set, ensuring no image overlap. Prior to allocation, radiographs from the holdout test set were randomly selected from patients with all the medical data necessary for calculating the 10-year ASCVD risk score to facilitate comparison between the algorithm and traditional cardiovascular risk factors. None of the patients in the holdout test set were included in the training and validation sets to minimize data leakage. Radiographs associated with partial calcium scoring data were excluded from the experiment if the relevant label data were missing. Using Keras 2.1.6 (https://keras.io/) with a Tensorflow 1.12.0 (https://www.tensorflow.org/) backend in Python 3.6, an attention-based network architecture was built on the VGG-16 DCNN, pretrained on ImageNet weights. This design followed an open source network architecture that was built for pediatric bone age assessment by Mader with code accessible online (13). Design was similar to those used in winning algorithms in the RSNA bone age challenge (10). Network adjustments were made only to the top-level layers for training with preservation of original ImageNet weights. The final network layer was modified for binary prediction of (a) zero versus nonzero total calcium scores, (b) the presence or absence of calcium in individual coronary artery territories, and (c) CAC above or below absolute total calcium score thresholds of 100 and 300.

Separate models were individually trained for each of these classifications for both the posteroanterior and the lateral chest radiographs. Hyperparameters were set to minimize validation loss using binary cross-entropy with a batch size of 32, maximum epochs of 200, and early stopping to avoid overfitting. Data augmentation was performed during each epoch using randomly applied shift, rotation, skew, and scale. For lateral radiographs, horizontal flip augmentation was also applied. All computation was performed on a workstation with an NVIDIA GTX 1060 graphics processing unit.

Statistical Analysis

Binary classifier performance on the holdout test set was measured using area under the receiver operating characteristic curve (AUC) with calculations of sensitivity, specificity, positive predictive value, and negative predictive value at varying thresholds to attain target sensitivity or specificity of approximately 90%. This was done for each of the classification models that was trained for the total score and vessel-specific assessments for both the posteroanterior and lateral radiographs. Machine learning statistical analysis was performed using the Python scikit-learn metrics library version 0.23 for quantifying quality of machine learning predictions (14). AUCs were compared using the DeLong parametric method. Ninety-five percent CIs were generated using two-sided CIs for proportions. Heat maps were produced to highlight image features emphasized by the DCNN in its decision-making, on the basis of the attention mask layer generated from the input image (13).

ASCVD 10-year cardiovascular risks were calculated for each of the patients in the test set. Risk scores for the cohort of patients who were predicted as having a nonzero calcium score on frontal radiographs were compared with those who were predicted as having a zero calcium score by two-tailed t test with Microsoft Excel. Multivariate logistic regression was also performed on the test set using traditional ASCVD risk factors as well as the algorithm prediction to assess independent correlation with a nonzero calcium score. Logistic regression was performed using SPSS Statistics version 16.0. Statistical significance was determined by P values less than .05.

Finally, we reviewed the radiologist-dictated reports from the chest radiographs in our data set and determined the frequency of mention of coronary artery disease or CAC in the original chest radiograph reports.

Results

Calcium Score Demographics

In our study population, total calcium scores ranged from 0 to 9052.8 (median: 9.7; Q1–Q3: 0–97.3) (Table 1); of these, 44% (681 of 1543 radiographs for whom total calcium scores could be computed) were negative for coronary calcium with scores of 0 (Fig 2), providing an approximately balanced distribution between zero and nonzero total calcium scores.

Figure 2:

Histogram demonstrates the distribution of total calcium scores in the data set. Approximately 44% of radiographs (681 of 1543 for which total calcium scores could be computed) were associated with a calcium score of 0.

Review of chest radiograph radiology reports showed no prospective identification of coronary artery disease or calcifications on chest radiographs obtained prior to CT scans. Coronary calcification was reported in only two chest radiographs, which were both obtained after the calcium score was available (scores of 199.0 and 270.8). The data set included 888 radiographs where the corresponding CT had a nonzero calcium score and 382 with a calcium score greater than 100.

Binary Classification of Zero versus Nonzero Total Agatston Scores

Binary classification between zero and nonzero Agatston scores attained an AUC of 0.73 on posteroanterior radiographs (Fig 3). An algorithm optimized for sensitivity could attain a sensitivity of 91% with 29% specificity (Table 2). Optimized for specificity, the algorithm could attain specificity of 88% with 30% sensitivity.

Figure 3:

Receiver operating characteristic curve for the binary classification of zero and nonzero Agatston scores on frontal chest radiographs, demonstrating an area under the curve (AUC) of 0.73.

Table 2:

Algorithm Performance for Prediction of Coronary Artery Calcium on Frontal Radiographs at Various Optimizations

The cohort of patients who were predicted as having a nonzero score on frontal radiographs had a higher 10-year ASCVD risk of 17.2% ± 10.9 compared with 11.9% ± 10.2 for those who were predicted as having a calcium score of 0 (P < .001; Table 3). Multivariate logistic regression demonstrated a positive score on the algorithm to be a statistically significant predictor of a nonzero calcium score independent of traditional ASCVD risk factors, with an odds ratio of 2.48 (95% CI: 1.32, 4.64) (Table 4).

Table 3:

Comparison of Patients Who Tested Negative versus Positive on a DCNN in Predicting Likelihood of Nonzero Calcium Score on Frontal Chest Radiographs

Table 4:

Multivariate Logistic Regression

Estimation of total CAC on lateral radiographs achieved an AUC of 0.70, with the performance similar to the model trained on posteroanterior radiographs (AUC, 0.73; P = .56) (Table 4).

Classification of Presence or Absence of Calcium in Individual Vessels

In per-vessel analysis, the highest burden of coronary calcium seen at calcium scoring CT was found in the left anterior descending coronary artery (43.9% of the total Agatston scores among all radiographs), followed by the right coronary artery (33.0%) (Table 1). When binary classification was restricted to assessing scores in individual vessels, classification performance tended to decrease. Performance was slightly higher for assessing the right coronary artery on posteroanterior radiographs with an AUC of 0.71 (Fig 1) and the left circumflex on lateral radiographs with an AUC of 0.75, though these differences were not statistically significant (Table 5).

Table 5:

AUCs of Binary Algorithms

Classification of High and Non-High CAC Thresholds

When trained on binary classification between absolute calcium score thresholds, algorithms trained on frontal radiographs attained an AUC of 0.74 for a threshold of 100 and an AUC of 0.70 for a threshold of 300.

Attention Heat Maps

Attention maps created from the highest performing networks primarily localized to the cardiac silhouette (Fig 4), though they often included secondary findings in decision-making. For example, the presence of a cardiac defibrillator strongly biased the algorithm to output “positive” for a nonzero calcium score (Fig 5) rather than examining the cardiac silhouette.

Figure 4a:

Attention-based heat maps produced when training models on (a) frontal and (b) lateral chest radiographs labeled with total Agatston scores. Despite training only on calcium score numbers, algorithms learned to focus on and prioritize the cardiac silhouette to predict the presence of coronary artery calcium. Left column, radiographs; middle column, attention-based heat maps; right column, heat maps overlaid on radiographs.

Figure 5:

Secondary findings associated with coronary artery disease, such as the presence of a cardiac defibrillator, strongly biased the algorithm, as shown in these two examples. Left column, radiographs; middle column, attention-based heat maps; right column, heat maps overlaid on radiographs.

Figure 4b:

Attention-based heat maps produced when training models on (a) frontal and (b) lateral chest radiographs labeled with total Agatston scores. Despite training only on calcium score numbers, algorithms learned to focus on and prioritize the cardiac silhouette to predict the presence of coronary artery calcium. Left column, radiographs; middle column, attention-based heat maps; right column, heat maps overlaid on radiographs.

Algorithm Processing Time

The machine learning algorithms demonstrated rapid processing time using the specified hardware, with analysis of a sample test set of 241 images taking approximately 2 seconds to perform.

Discussion

Machine learning using DCNNs has demonstrated the potential to identify findings on medical images that are often overlooked by human observers. In our study, we demonstrate that DCNNs can predict the likelihood of CAC on chest radiographs, albeit with modest performance, and that this correlates with patient-specific cardiovascular risk.

Machine learning has previously been used in the automated calculation of Agatston scores from electrocardiographically gated cardiac CT scans (15), as well as providing accurate estimates from nongated CT studies (16,17). However, CT scans have higher cost and radiation than chest radiographs, which are obtained more commonly. Chest radiographs are not only used in the evaluation of suspected cardiac pathologic conditions but for a variety of diagnostic purposes, opening the possibility for opportunistic screening. An automated method to screen for CAC on chest radiographs may provide clinically meaningful prognostic information in patients with unsuspected coronary artery disease. As chest radiographs are routinely obtained in the initial evaluation of chest pain in the emergency department (18), an algorithm that predicts the presence of CAC may be useful in guiding subsequent diagnostic evaluation in an acute setting.

Examining our study population, we found that nearly half of the calcium scoring studies had an Agatston score of 0, while others demonstrated calcium scores as high as 9052.8. Current guidelines recommend only testing patients at an intermediate risk and not those already at low or high cardiovascular risk where testing may be more harmful than beneficial due to unnecessary cost, radiation exposure, and need to work up incidental findings that may be of no clinical significance (1). The high prevalence of patients with very low scores raises the question if many of these cardiac CT examinations could be avoided if there were a way to prescreen for low calcium scores. Chest radiographs enhanced by deep learning algorithms, such as those described in our study, could potentially act as an intermediary screening test for patients at low or low-intermediate risk of coronary artery disease to determine if a calcium score CT would be of value. Such a strategy using deep learning algorithms could improve the pretest probability and enhance the diagnostic yield of the test. In select patients, an algorithm with high sensitivity could be used to safely exclude patients unlikely to have CAC on chest radiographs and thereby reduce the number of unnecessary calcium score CT examinations performed. For example, in the test set of 241 radiographs, 54 (22%) had true-negative findings and may not have needed full calcium scoring CT.

Our results also demonstrate that DCNNs on radiographic images have value in predicting cardiovascular risk by correlating with ASCVD 10-year risk, which is a well-established clinical method of predicting cardiovascular risk and guiding therapeutic management (2,4). Furthermore, we analyzed risk thresholds that have been demonstrated by another study to be associated with significantly higher risks of “hard” cardiac events such as myocardial infarction and death (3,19). Cardiovascular risk scores require a specific and complete set of demographic information, medical history, and laboratory and physical examination data; if any information is missing, the risk cannot be calculated. DCNNs like those developed in our study could provide meaningful cardiovascular risk stratification in settings when this information may not be readily available, such as in the emergency department when medical history is often incomplete. Furthermore, incorporation of image-based data, such as features extracted by DCNNs, into such risk calculators may also improve the accuracy of disease prognostication or risk prediction. We found that the DCNN predictions from chest radiographs correlated with CAC independent of traditional risk factors, suggesting this could provide supplementary information in risk assessment models.

We found that performance tended to decrease when predicting calcifications in specific coronary vessels, likely related to the smaller size and more detailed analysis needed for making an accurate prediction, and that difference depended on radiograph projection. Our algorithm performed relatively better for the right coronary artery on posteroanterior radiographs and left circumflex artery on lateral radiographs, which we hypothesize to be due to the better visibility of each vessel along their long axis in the corresponding view (Fig 1). A larger data set to improve individual vessel performance can have implications in assisting with clinical evaluation and increasing the specificity of electrocardiography findings. For example, in the acute setting, a patient with chest pain and electrocardiography findings of inferior wall ischemia could substantially benefit from an algorithm that can reliably identify CAC in the right coronary artery.

Several limitations are noted. Most importantly, the inherent “black box” nature of deep learning means that the algorithm is using all available radiographic information to predict the presence of CAC, and its output may not necessarily correspond to a clear presence of coronary calcification on chest radiographs that a radiologist would expect to see. Upon review of attention maps, extracoronary confounders—particularly cardiac defibrillators, which were seen in approximately 61 images (4% of the data set)—were sometimes more important for the algorithm's holistic decision-making rather than visualization of CAC. Potentially, the algorithm recognized the correlation between defibrillators (indicating underlying cardiac disease) and CAC and used this association to correctly predict the presence of CAC. Occasionally, atherosclerosis in other locations was also highlighted, which has been shown on chest radiographs to be an independent predictor of cardiovascular events (20). This would also be true for factors like the presence of mitral annular or valvular calcifications that the algorithm was not specifically trained to recognize or exclude. It has been shown that the presence of such valvular calcifications is correlated with severe CAC (21), and this can also serve as an extracoronary confounder that would bias the decision-making of the algorithm. In such cases, the algorithm may still produce a correct prediction but for a confounding explanation. This affects the clinical value of such algorithms and is a limitation to recognize with machine learning approaches to solutions. However, in our analyses, we demonstrated that a positive score on the algorithm does independently correlate with a nonzero calcium score and is associated with increased cardiovascular risk. An additional limitation was the downsampling of images to accommodate the input designs of the pretrained DCNNs used in this study, which resulted in the loss of anatomic detail related to the coronary arteries. This limitation is important given the relatively small size of the vessels, as well as the subtle nature of CACs.

A number of factors also biased our algorithm. The study population was heavily weighted toward our community demographics, with higher representation of African American and Black patients and a lower representation of Asian and Hispanic patients, and this likely biased calcium score distribution when correlated with known epidemiologic data (2). However, our data set demonstrated median MESA scores close to the 50th percentile when accounting for patient demographics (Table 1). The inclusion of radiographs from a variety of imaging sites at our institution with different examination parameters and limitations likely also negatively affected algorithm performance.

There are several directions to improve network performance. Future study could include higher-resolution images and adapt DCNN architectures to accommodate such higher-resolution images. A larger data set would likely also greatly improve the performance of the algorithm. More data and external validation would certainly be required for any clinical use of the algorithm. We additionally experimented with training a regression algorithm to predict continuous calcium scores but found that a much larger data set would be required for adequate performance for prediction of numeric calcium scores.

Our study serves as a proof-of-concept for DCNNs to predict CAC and cardiovascular risk from chest radiographs, which is often beyond human radiologist attention, as indicated by the lack of radiologist reporting of CAC on the original radiograph reports. These results serve as a pilot for future applications in deep learning in radiology that extends beyond the common use of deep learning to detect findings already noted by radiologists. Here we demonstrate the ability to extract information from medical imaging not readily noted on human review that may affect clinical management and diagnostic workup.