Prediction of Revascularization by Coronary CT Angiography using a Machine Learning Ischemia Risk Score

Abstract

Objectives:

The Machine Learning Ischemia Risk Score (ML-IRS) is a machine-learning based algorithm designed to identify hemodynamically significant coronary disease using quantitative coronary computed tomography angiography (CCTA). The purpose of this study was to examine whether the ML-IRS can predict revascularization in patients referred for invasive coronary angiography (ICA) after CCTA.

Methods:

This study was a post-hoc analysis of a prospective dual-center registry of sequential patients undergoing CCTA followed by ICA within three months, referred from inpatient, outpatient, and emergency department settings (n=352, age 63±10 years, 68% male). The primary outcome was revascularization by either percutaneous coronary revascularization or coronary artery bypass grafting. Blinded readers performed semi-automated quantitative coronary plaque analysis. The ML-IRS was automatically computed. Relationships between clinical risk factors, coronary plaque features, and ML-IRS with revascularization were examined.

Results:

The study cohort consisted of 352 subjects with 1056 analyzable vessels. The ML-IRS ranged between 0-81% with a median of 18.7% (6.4-34.8). Revascularization was performed in 26% of vessels. Vessels receiving revascularization had higher ML-IRS (33.6%(21.1-55.0) versus 13.0%(4.5-29.1), p<0.0001), as well as higher contrast density difference, and total, non-calcified, calcified, and low-density plaque burden. ML-IRS, when added to a traditional risk model based on clinical data and stenosis to predict revascularization, resulted in increased area-under-the-curve from 0.69(95% CI:0.65-0.72) to 0.78(95% CI:0.75-0.81),p<0.0001); with an overall continuous net reclassification improvement of 0.636(95% CI:0.503-0.769),p<0.0001.

Conclusions:

ML-IRS from quantitative coronary CT angiography improved the prediction of future revascularization and can potentially identify patients likely to receive revascularization if referred to cardiac catheterization.

Introduction

The diagnostic yield of invasive coronary angiography (ICA) has been a well-documented concern in cardiology. A major study in 2010 revealed that only 37% of patients receiving ICA were found to have stenosis of ≥50%.[1] Coronary CT angiography (CCTA) has emerged as a powerful tool for assessment of patients with suspected coronary artery disease (CAD).[2] It has grown into an increasing role as a gatekeeper for referral to ICA[3] but further optimization is needed. The PROMISE trial showed that CCTA resulted in a higher rate of ICA than functional testing in symptomatic patients with suspected CAD. Furthermore, a coronary intervention was performed only half of the time despite stenosis of ≥50% on ICA was usually present.[4] In order to improve CCTA’s role as a gatekeeper, a method to safely and effectively decrease the proportion of patients referred for ICA who do not receive intervention is essential.

An algorithm called the Machine Learning Ischemia Risk Score (ML-IRS) has been previously developed.[5] It used automated feature selection followed by machine learning integration to predict lesion-specific ischemia defined by invasive fractional flow reserve (FFR) ≤ 0.8. The machine learning model incorporates quantitative CCTA factors and demographic information to generate a percent probability of pathological FFR on a per-vessel basis. It has been shown to improve prediction of lesion-specific ischemia over traditional measurements and is integrated into coronary plaque analysis research software for application to new data.[5]

Patients receiving CCTA who are subsequently referred for cardiac catheterization, but do not receive revascularization, are a high-cost population with low yield from the invasive procedure; and it would therefore be beneficial to effectively identify these patients. The purpose of this study was to examine whether the ML-IRS from quantitative CT can predict revascularization in patients referred to ICA after CCTA, in a prospective dual-center registry.

Materials and Methods

Study Population:

The study cohort included 352 consecutive patients (age 63±10 years, 68% male) between 2013 and 2017 at two advanced cardiac care facilities within the United States (Cedars Sinai Medical Center, Los Angeles, CA; Oklahoma Heart Institute, Tulsa, OK). The cohort selected patients who were referred for standard clinical CCTA followed by ICA due to decision by a primary treating physician within 3 months. Patients with incomplete clinical data, poor image quality, coronary calcium score (CCS) >1000 Agatston units (AU), or prior revascularization by coronary artery bypass grafting or percutaneous coronary intervention were excluded. IRB approval was obtained at both institutions. Written informed consent was obtained for all patients. There was no overlap in subjects from any prior publications.

Clinical data including age, sex, body mass index, history of hypertension, hyperlipidemia, current tobacco use, family history of CAD, symptoms of chest pain or shortness of breath and referral location (inpatient, emergency department, or outpatient) were obtained by database or chart review. Procedural reports from cardiac catheterization were extracted for all patients. The report was analyzed for operator-determined estimate of vessel stenosis, any intervention performed, fractional flow reserve measurement, and intravascular ultrasound measurement. Patients who received percutaneous coronary intervention or were referred for coronary artery bypass grafting as stated in the procedure report were counted as having received revascularization.

CT Acquisition:

CT scans were performed on multiple dual-source CT scanner platforms (Siemens Definition, Flash, and Force scanners, Siemens Heathineers), using standard clinical coronary imaging protocols as previously described.[6] In patients without contraindications, sublingual nitroglycerin was administered. Beta blockade with metoprolol was given prior to imaging with primary heart rate goal of ≤ 65 bpm. Iohexol (Omipaque 350mg/mL, GE Healthcare) was administered via large-bore intravenous catheter. Contrast volume (90 to 140 mL) and injection rate (5 to 9 mL/s) were adapted to body surface area and followed by saline flush. Timing bolus or automated bolus tracking at the descending aorta was used to trigger acquisition. Whole-volume image acquisition was completed in a single breath-hold. The patients did not receive FFRct analysis following CCTA.

CT Analysis

Standard clinical measurement of CCS by Agatston score was performed and recorded at the time of CCTA. Quantitative image analysis was performed by two readers (A.K., over 3 years’ experience and P.A.M., over 1.5 years’ experience with cardiac CT) who were blinded to clinical characteristics, ICA data, and previous clinical CCTA read. Plaque analysis was performed in all vessels with distal normal reference ≥ 2.0 mm using semi-automated software (Autoplaque version 2.5, Cedars-Sinai Medical Center). Readers identified regions with visible disease. Vessel lumen and wall contours were generated, followed by manual adjustment as needed. Plaque characterization was performed automatically using adaptive scan-specific attenuation thresholds. Low-density non-calcified plaque was defined as HU < 30, as described previously. Coronary segments with uncertainty in plaque presence were reviewed and adjudicated by two readers. Approximate time for analysis per case was 20 minutes.

Vessel parameters including maximal diameter stenosis, plaque volumes, plaque burdens, and contrast density difference (CDD) were calculated automatically after completion of quantitative semi-automated plaque analysis. Quantitative stenosis was calculated by dividing the narrowest luminal diameter by the mean of two normal non-diseased reference points. Significant stenosis was defined by quantitative diameter stenosis ≥ 70%. Plaque burden was defined as plaque volume normalized by total diseased vessel volume. CDD over a lesion was computed as follows: the luminal contrast density, defined as mean luminal attenuation per unit area, was computed automatically over 1-mm cross-sections of the arterial segment. The CDD was defined as the maximum percentage difference in contrast densities, relative to the proximal reference cross-section with no disease.

Machine-Learning Ischemia Risk Score

The ML-IRS is automatically generated as the probability (%) of lesion-specific ischemia (invasive FFR≤0.8) for the left anterior descending, left circumflex, and right coronary arteries. The ML-IRS was previously developed in an independent population, using a supervised learning process by feature selection with information gain ranking and ensemble classification boosting with ten-fold cross-validation to predict invasive lesion-specific ischemia. It includes factors based on quantitative CT variables (CDD, maximum quantitative stenosis, minimum luminal diameter, low-density non-calcified plaque volume, total non-calcified plaque volume, plaque length, total plaque volume, vessel volume, minimal luminal area, low-density non-calcified plaque composition, maximum area stenosis, low-density non-calcified plaque burden, myocardial mass, non-calcified plaque burden, total plaque burden, non-calcified plaque composition, and maximum remodeling index) plus age and gender of the patient obtained from the image DICOM header.[5]

The primary endpoint in our study was revascularization during or as a direct referral after ICA. Secondary analysis was prediction of lesion-specific ischemia (FFR≤0.80) and prediction of revascularization within the sub-cohort of patients who received invasive FFR during ICA.

Statistical Analysis

Continuous variables were expressed as a mean and standard deviation when normally distributed and median and interquartile range when not. Univariable and multivariable analyses were performed on both a per-patient and per-vessel basis. Per-vessel analyses included vessel-specific quantitative plaque features, ML-IRS, and whether revascularization was performed in the vessel. Per-patient analyses included total quantitative plaque features, highest ML-IRS, and whether revascularization was performed in any vessel in the patient. Univariable analysis used two-sample Wilcoxon rank-sum tests for non-parametric values, chi- squared test for categorical variables, and Student’s T test for normally distributed numerical variables. Simple logistic regression was used for FFR subgroup analysis. Multivariable logistic regression was used to assess associations between clinical risk factors (age, sex, presence of symptoms, history of hypertension, hyperlipidemia, or diabetes, and family history of CAD), significant stenosis, patient referral location (emergency department or inpatient versus outpatient) and ML-IRS with revascularization. Optimized cutoff for ML-IRS was calculated on a per-vessel basis using the two-graph ROC analysis as the intersection of the sensitivity and specificity graphs.[7] Elements with missing data were excluded from logistic regression analysis.

Three multivariable models were created to assess the ability of the ML-IRS to improve prediction compared with other methods. Model 1 represented clinical risk assessment and included age, sex, referral location, symptoms, hypertension, hyperlipidemia, diabetes, current tobacco use, and family history. Model 2 represented a clinical risk assessment combined with traditional use of CCTA and combined the covariates in Model 1 plus presence of significant stenosis by CCTA. In Model 3 we added ML-IRS to the covariates in Model 2 to assess whether ML-IRS could improve on traditional prediction models. ROC curves were generated for the models and AUC’s were compared. Incremental predictive value between Models 2 and 3 was also measured using continuous net reclassification improvement (NRI), as pre-existing established risk categories were not known for the ML-IRS.[8; 9] All statistical analyses were performed with Stata version 15.1. The NRI computation was performed using a SAS v9.4 module. All p-values <0.05 were considered significant.

Results

A total of 352 patients who received CCTA followed by ICA were identified and analyzed, with 1056 total vessels included in the analysis (Table 1). The mean age was 63.2 ± 10.4 years and 67.6% of the patients were male. Most patients (85.1%) presented with symptoms. The median body mass index (BMI) was 27.5 kg/m² (24.7-31.8) and the median coronary calcium score by Agatston was 264.7 (82.0-533.1). The FFR subpopulation consisted of 74 patients with 83 analyzable vessels and had similar patient demographics compared to the overall population (Table 1). Figures 1 and 2 show case examples from our patient population.

Table 1.

Demographic information for total cohort and for sub-cohort of patients with fractional flow reserve

	Total Population	Revascularization	No Revascularization	p-value
Total Patients	352	208 (59%)	144 (41%)
Age (years)	63.2±10.4	62.6±10.5	64.1±10.3	0.19
Sex (male)	238 (67.6%)	148 (71.2%)	90 (62.5%)	0.09
Symptomatic	298 (85.1%)	171 (48.9%)	127 (36.3%)	0.11
Hypertension	227 (64.5%)	131 (63.0%)	96 (66.7%)	0.48
Hyperlipidemia	198 (56.2%)	116 (55.8%)	82 (56.9%)	0.83
Diabetes	93 (26.4%)	57 (27.4%)	36 (25.0%)	0.62
Tobacco	56 (15.9%)	36 (17.3%)	20 (13.9%)	0.39
Family CAD	180 (51.1%)	113 (54.3%)	67 (46.5%)	0.15
Inpatient/ED	260 (74.9%)	158 (77.5%)	102 (71.3%)	0.2
BMI (kg/m2)	27.6 (24.7-31.8)	27.3 (24.7-31.8)	28.0 (24.5-31.8)	0.47
CCS	262 (82.0-533)	299 (91.8-550)	216 (68.6-530)	0.14
Site (from site 1)	226 (64.2%)	138 (66.3%)	88 (61.1%)	0.37
Patients witd FFR	74 (21.1%)	47 (63.5%)	27 (36.5%)
Age (years)	63.5±9.7	62.9±10.2	63.3±9.0	0.89
Sex (male)	49 (66.2%)	33 (70.2%)	16 (59.3%)	0.48
Symptomatic	62 (84.9%)	39 (84.8%)	23 (85.2%)	0.96
Hypertension	48 (64.9%)	31 (66.0%)	17 (63.0%)	0.8
Hyperlipidemia	43 (58.1%)	27 (57.4%)	16 (59.3%)	0.88
Diabetes	17 (23.0%)	9 (19.1%)	8 (29.6%)	0.3
Tobacco	11 (14.9%)	8 (17.0%)	3 (11.1%)	0.49
Family CAD	40 (54.1%)	26 (55.3%)	14 (51.9%)	0.77
Inpatient/ED	64 (86.5%)	40 (85.1%)	24 (88.9%)	0.65
BMI (kg/m2)	27.6 (24.2-30.9)	27.2 (24.5-30.8)	28.2 (23.6-30.8)	0.99
CCS	263.1 (88.7-530.9)	315.7 (120.2-524.0	154.0 (73.6-580.5)	0.25
Site (from site 1)	56 (76%)	37 (78%)	19 (70%)	0.6

^*Results expressed as mean ± standard deviation, frequency (percent), or median (interquartile range). BMI: Body mass index, CAD: Coronary artery disease, CCS: Coronary calcium score, ED: Emergency Department referral location. FFR: Fractional flow reserve.

Figure 1:

Case example: Machine Learning Ischemia Risk Score (ML-IRS) and coronary plaque analysis in a 67-year old male symptomatic patient undergoing coronary CT Angiography, left anterior descending (LAD) artery. Panel A: Sample output of quantitative analysis and ML-IRS. Panel B: Multiplanar oblique views of LAD coronary atherosclerosis (red: non-calcified plaque, yellow: calcified plaque). Panel C: Curved multiplanar reformat of LAD atherosclerosis. ML-IRS in the LAD was 60%, invasive fractional flow reserve measured in the LAD was 0.73 (ischemic).

Figure 2:

Case example: Machine Learning Ischemia Risk Score (ML-IRS) and coronary plaque analysis in a 57-year old male symptomatic patient undergoing coronary CT Angiography, left anterior descending (LAD) artery. Panel A: Sample output of quantitative analysis and ML-IRS. Left circumflex artery had no coronary plaque and therefore risk was not estimated. Panel B: Multiplanar oblique views of LAD coronary atherosclerosis (red: non-calcified plaque, yellow: calcified plaque). Panel C: Curved multiplanar reformat of LAD atherosclerosis. ML-IRS in the LAD was 11%, invasive fractional flow reserve measured in the LAD was 0.83 (not ischemic).

Per-Patient Univariable Analysis

Revascularization was performed in 59% of the total subjects (Table 1). There was a higher proportion of male gender, symptoms, and family history of CAD in patients receiving revascularization in the total population. In the sub-population with FFR performed there was a higher proportion of men, higher CCS, and a lower proportion of patients with diabetes. None of these differences met statistical significance and may be partially or wholly due to uncontrolled biases. There was no relation between site (Cedars Sinai versus Oklahoma Heart Institute) and revascularization.

Per-Vessel Univariable Analysis

In the per-vessel analysis, the ML-IRS ranged between 0 to 81% with a median of 18.7% (6.4-34.8) (Table 2). Revascularization was performed in 26% of the vessels. Univariable analysis found that vessels receiving revascularization had higher ML-IRS (33.6% (21.1-55.0) versus 13.0% (4.5-28.1), p<0.0001). Revascularized vessels also had higher CDD, total plaque burden, non-calcified plaque burden, low-density plaque burden, and calcified plaque burden. Interquartile ranges of plaque components are noted to overlap between revascularization and non-revascularization groups despite statistical significance. This suggests that while there are differences between groups, individual plaque features alone are insufficient to predict revascularization.

Table 2.

Per-vessel plaque features in relation to revascularization in all vessels, and in the subgroup with measured FFR.

	Total Population	Revascularization	No Revascularization	p-value
Total Vessels	1056	271 (26%)	785 (74%)
Significant Diameter Stenosis > 70%	135 (12.8%)	84 (31%)	51 (6.5%)	<0.0001
Machine-Learning Ischemia Risk Score	18.7% (6.4-34.8)	33.6% (21.1-55.0)	13.0% (4.5-28.1)	<0.0001
Total Plaque Burden	47.8% (36.1-56.4)	54.5% (47.2-60.7)	44.9% (38.0-54.3)	<0.0001
Non-Calcified Plaque Burden	44.4% (33.1-52.3)	49.6% (42.7-57.6)	42.0% (24.8-50.7)	<0.0001
Low-Density Non-Calcified Plaque Burden	5.6% (2.1-8.8)	7.9% (5.3-10.7)	4.3% (0.8-7.9)	<0.0001
Calcified Plaque Burden	1.5% (0-4.1)	2.8% (0.7-5.3)	1.1% (0-3.4)	<0.0001
Contrast Density Difference	18.5 (5.2-29.8)	27.1 (18.9-37.0)	14.1 (0-25.6)	<0.0001
Vessels with FFR	83 (7.9%)	41 (49.4%)	42 (50.6%)
Significant Diameter Stenosis > 70%	15 (18%)	9 (10.8%)	6 (7.2%)	0.364
Average FFR	0.80±0.10	0.74±0.08	0.85±0.07	<0.0001
Machine-Learning Ischemia Risk Score	31.4% (18.7-48)	36.0% (25.5-50.7)	21.5% (14.9-43.2)	0.006
Total Plaque Burden	52.4% (47.0-58.2)	54.1% (47.6-59.2)	50.3% (46.2-58.2)	0.420
Non-Calcified Plaque Burden	48.2% (41.6-55.2)	49.2% (41.0-55.4)	47.2% (43.7-54.8)	0.800
Low-Density Non-Calcified Plaque Burden	7.2% (4.3-9.8)	7.9% (5.8-10.4)	6.4% (3.6-8.5)	0.070
Calcified Plaque Burden	2.4% (0.9-6.1)	2.9% (1.8-6.2)	2.1% (0.6-5.7)	0.140
Contrast Density Difference	24.0 (16.5-33.7)	26.2 (21.1-37.7)	21.6 (10.6-32.4)	0.016

^*Results expressed as mean ± standard deviation, frequency (percent), or median (interquartile range). FFR: Fractional flow reserve.

In the FFR sub-cohort (83 vessels, 74 patients) the average FFR was 0.80 ± 0.10, with an average of 0.74 ± 0.08 in vessels receiving revascularization versus 0.85 ± 0.07 in vessels not revascularized (p<0.0001) (Table 2). Only ML-IRS and CDD were significantly different between patients who received revascularization versus those who did not in this sub-population. Overlap of interquartile range is again seen; this overlap may be reduced by sampling higher numbers. ML-IRS in patients with FFR≤0.8 was 38.5% (25.6-50.6) versus 21.3% (12.9-31.8) for FFR>0.8, p=0.0007 for difference. Univariable logistic regression for ML-IRS’s association with invasive FFR≤0.8 was significant. Odds of FFR≤0.8 was increased by 55% for every 10% increase in ML-IRS (p=0.003). Similarly within the FFR population, odds of revascularization increased by 39% for every 10% increase in ML-IRS (p=0.016).

Multivariable Analysis

Per-vessel and per-patient logistic multivariable analyses were performed for prediction of revascularization (Table 3). Variables included were age, sex, symptoms (as defined by chest pain or shortness of breath), clinical risk factors (hypertension, hyperlipidemia, diabetes, current tobacco use, and family history), significant stenosis > 70% (present in any vessel for per-patient analysis), referral location (inpatient or emergency department versus outpatient), and ML-IRS (maximum score for per-patient analysis). In the per-vessel analysis, a 10% increase in ML-IRS resulted in a 46% increase in odds of revascularization (p<0.0001). In the per-patient analysis, odds of revascularization increased by 20% per 10% increase in ML-IRS (p=0.011). If significant stenosis was present, odds increased by 98% (p=0.007) in the per-vessel and 94% (p=0.038) in the per-patient analysis. Presence of family history of CAD increased odds of revascularization by 49% (p=0.019) in per-vessel and 74% (p=0.025) in per-patient analysis. Symptoms were significant in the per-vessel analysis but not the per-patient analysis. Unexpectedly, the presence of symptoms appeared protective (OR 0.64 with symptoms, p=0.039). Referral location was associated with an increased odds of revascularization by 76% for patients from inpatient or emergency department in the per-patient analysis (p=0.042). Per-vessel ML-IRS had an optimized cutoff within the population at a value of 23.9%, which yielded an odds ratio for revascularization of 5.5 (4.0-7.4), a sensitivity of 73.2% and specificity of 68.0%, positive predictive value of 44.2%, negative predictive value of 88.0%.

Table 3.

Per-vessel and Per-patient multivariable analyses for predicting revascularization including clinical risk factors, significant stenosis, and Machine Learning Ischemia Risk Score (per 10% change).

Per Vessel	Odds Ratio	95% CI	B	p-value
Age (year)	0.998	(0.982-1.014)	−0.002	0.779
Male Sex	0.777	(0.543-1.113)	−0.252	0.169
Symptomatic	0.64	(0.418-0.979)	−0.447	0.039
Hypertension	0.769	(0.551-1.073)	−0.263	0.122
Hyperlipidemia	1.063	(0.771-1.466)	0.061	0.708
Diabetes	1.074	(0.755-1.529)	0.072	0.691
Tobacco	1.058	(0.694-1.612)	0.056	0.794
Family CAD	1.486	(1.069-2.065)	0.396	0.019
Referral Location	1.212	(0.842-1.745)	0.192	0.301
Significant Stenosis	1.981	(1.201-3.263)	0.683	0.007
ML-IRS (per-10%)	1.464	(1.338-1.602)	0.381	<0.001
Per Patient	Odds Ratio	95% CI	B	p-value
Age (year)	1.003	(0.980-1.027)	0.003	0.815
Male Sex	1.042	(0.618-1.760)	0.042	0.876
Symptomatic	0.523	(0.262-1.043)	−0.648	0.065
Hypertension	0.666	(0.405-1.095)	−0.407	0.109
Hyperlipidemia	1.041	(0.648-1.674)	0.041	0.866
Diabetes	1.131	(0.664-1.927)	0.123	0.651
Tobacco	1.583	(0.830-3.016)	0.459	0.162
Family CAD	1.744	(1.072-2.835)	0.556	0.025
Referral Location	1.761	(1.019-3.043)	0.566	0.042
Significant Stenosis	1.939	(1.035-3.630)	0.662	0.038
Maximum ML-IRS (per-10%)	1.211	(1.192-1.230)	0.191	0.011

^*CAD: Coronary artery disease, ML-IRS: Machine learning ischemia risk score, CI: Confidence Interval

Multivariable logistic regression models to predict revascularization were compared. Model 1, (age, sex, referral location, symptoms, hypertension, hyperlipidemia, diabetes, current tobacco use, and family history) assessed clinical risk. The AUC for prediction of revascularization for Model 1 was 0.58 (0.54-0.62). Model 2 (Model 1 plus presence of significant stenosis) represented addition of traditional use of CCTA. The AUC for Model 2 was 0.69 (0.65-0.72). Model 3 (Model 2 plus ML-IRS) assessed the additional importance of the ML-IRS. The AUC for Model 3 was 0.78 (0.75-0.81). There was a significant improvement between each model (p < 0.0001) (Figure 3). The continuous NRI for addition of ML-IRS to standard CCTA analysis (Model 3 over Model 2) was 0.636 (0.503-0.769 95% CI, p < 0.0001). This consisted of a 28% improvement in classification of patients receiving revascularization and a 36% improvement in those not receiving revascularization.

Figure 3:

Per-vessel prediction of revascularization with ROCs from multivariable models. P<0.0001 for difference between models. Model 1 (yellow): age, sex, symptoms, hypertension, hyperlipidemia, current smoking, diabetes, family history, and referral location; Model 2 (green): Model 1 + significant stenosis; and Model 3 (blue): Model 2 + ML-IRS.

Discussion

In this study, ML-IRS was significantly associated with revascularization in both univariable and multivariable analyses. Prediction of revascularization significantly improved when the ML-IRS was added to a model with clinical information and CCTA analysis including stenosis. This suggests that this on-site machine-learning based tool may be able to improve current practice patterns and predict which patients would benefit the most from referral to ICA. The continuous NRI measure also showed significant improvement with the integration of the ML-IRS with traditional methods of risk assessment; notably 36% improvement in predicting patients not receiving revascularization, indicating that addition of ML-IRS may help patients avoid unnecessary cardiac catheterization. In our population, only patients who were referred for ICA after CCTA were included. This is a high-risk population, as shown by the high rate of revascularization. Expansion to a lower-risk population of patients receiving CCTA may improve ability to avoid unnecessary referral. The analysis also showed low predictive ability for revascularization using traditional risk factors and symptoms. Presence of symptoms showed inverse association with revascularization, potentially due to the overall high prevalence of symptoms or patients without acute symptoms being specifically referred for revascularization as outpatients. The analysis of the FFR sub-population helps explain the method by which predictive ability may be applied to the general population. The ML-IRS was internally validated in the NXT trial population to predict invasive FFR,[5] which was also consistent in a subset of our dual-center population. While invasive FFR may be the gold standard for assessing physiological significance of a coronary stenosis, in our study design we recognize that the cost, need for medication administration, and logistics of performing FFR in all patients leads to a minority of patients receiving invasive assessment. Our study was structured to assess the patient-centered endpoint of revascularization: the clinical question facing patients with stenoses as well as their healthcare providers at the point of care. ML-IRS was more strongly associated with FFR than revascularization, which suggests that if invasive FFR had been used more broadly, revascularization rate may have been even more consistent with our predictions.

There have been multiple other approaches to artificial intelligence-based prediction of invasive FFR.[10-14], including commercially available noninvasive FFR (FFRct). FFRct uses computational fluid dynamics in a 3-D model of the coronary tree in order to provide an accurate prediction of invasive FFR from CCTA.[10; 15] While this approach has been successful, it incurs significant cost, and the need to send studies to an external location for analysis. Beyond luminal measurements alone, CTA also allows measurement of other features such as quantitative coronary plaque characteristics,[16; 17] and CDD[5], which are all included in the ML-IRS. Machine-learning based approaches have utilized quantitative CCTA to predict various outcomes including myocardial perfusion by PET[18] or CT,[19] or invasive FFR[14; 20] While these approaches have typically shown strong testing characteristics with relatively low processing times, to our knowledge, our study is the first to use a machine-learning based approach to predict revascularization. With the Youden’s index cutoff of 23.9%, the ML-IRS exhibited strong negative predictive value of 88.0%, despite the studied population being clinically challenging enough that all patients included received ICA after CCTA. This further suggests that use of the ML-IRS may help patients avoid unnecessary cardiac catheterization.

There are limitations in our study. Our population of patients receiving ICA after CCTA results in selection bias, and results may be different in all patients receiving CCTA. Additionally, only a portion of patients (21%) received invasive FFR. This limits the interpretability within the FFR sub-population. We note that our FFR utilization is consistent with real-world utilization of FFR in intermediate lesions.[21] FFR utilization in all patients may help support the efficacy of the algorithm; however, this may not reflect “real-world” practice of revascularization as shown in our study.[5; 22] The primary endpoint of revascularization does not capture long-term patient hard outcomes, which were not available to us at both sites. We did not directly compare our results to FFRct, since it was not available for all patients and could not be added for research purposes only. We recognize that this represents a separate, validated approach. While hypothesis-generating, our findings cannot make definitive statements regarding the ability of the ML-IRS to improve economic or clinical outcomes without prospective clinical studies.

In conclusion, ML-IRS from quantitative coronary CT angiography improved the prediction of future revascularization and can potentially identify patients likely to receive revascularization if referred to cardiac catheterization. This ML score is correlated with invasive FFR measurements providing dual-center external validation and improves on clinical risk prediction models.

Key Points:

• Machine learning ischemia risk from quantitative coronary CT angiography was significantly higher in patients who received revascularization versus those who did not receive revascularization.

• The machine learning ischemia risk score was significantly higher in patients with invasive fractional flow ≤ 0.8 versus those with > 0.8.

• The machine learning ischemia risk score improved the prediction of future revascularization significantly when added to a standard prediction model including stenosis.

Abbreviations:

BMI

Body Mass Index

CAD

Coronary Artery Disease

CCS

Coronary Calcium Score

CCTA

Coronary CT Angiography

CDD

Contrast Density Difference

FFR

Fractional Flow Reserve

ICA

Invasive Coronary Angiography

ML-IRS

Machine Learning Ischemia Risk Score

NRI

Continuous Net Reclassification Improvement