Diagnostic accuracy of a deep learning approach to calculate FFR from coronary CT angiography

Abstract

Background

The computational fluid dynamics (CFD) approach has been frequently applied to compute the fractional flow reserve (FFR) using computed tomography angiography (CTA). This technique is efficient. We developed the DEEPVESSEL-FFR platform using the emerging deep learning technique to calculate the FFR value out of CTA images in five minutes. This study is to evaluate the DEEPVESSEL-FFR platform using the emerging deep learning technique to calculate the FFR value from CTA images as an efficient method.

Methods

A single-center, prospective study was conducted and 63 patients were enrolled for the evaluation of the diagnostic performance of DEEPVESSEL-FFR. Automatic quantification method for the three-dimensional coronary arterial geometry and the deep learning based prediction of FFR were developed to assess the ischemic risk of the stenotic coronary arteries. Diagnostic performance of the DEEPVESSEL-FFR was assessed by using wire-based FFR as reference standard. The primary evaluation factor was defined by using the area under receiver-operation characteristics curve (AUC) analysis.

Results

For per-patient level, taking the cut-off value ≤ 0.8 referring to the FFR measurement, DEEPVESSEL-FFR presented higher diagnostic performance in determining ischemia-related lesions with area under the curve of 0.928 compare to CTA stenotic severity 0.664. DEEPVESSEL-FFR correlated with FFR (R = 0.686, P < 0.001), with a mean difference of –0.006 ± 0.0091 (P = 0.619). The secondary evaluation factors, indicating per vessel accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were 87.3%, 97.14%, 75%, 82.93%, and 95.45%, respectively.

Conclusion

DEEPVESSEL-FFR is a novel method that allows efficient assessment of the functional significance of coronary stenosis.

1. Introduction

Screening the functional significance of the coronary artery disease is critical to the medical decision-making.[1] However, there is a gap between the increasing population of the coronary arterial disease (CAD) and the application of clinical screening for preventing the major adverse clinical events (MACE). This is partially due to the invasive manner of the ‘gold standard’ examinations, such as invasive coronary angiography (ICA) and fractional flow reserve (FFR).[2] Non-invasive computed tomography (CT) has showed superior diagnostic performance in detecting the anatomic significance of coronary arterial stenosis [computed tomography angiography (CTA)[3] and CT perfusion[4] and hemodynamic significance using computed fluid dynamics (CFD) analysis.[5]^,[6] Despite the progressions in medical imaging technique and computation-aid diagnostic assistance have facilitated the non-invasive functional assessment of CAD,[7] the time-expense and excessive technical interferences have hindered the application in clinical routine. Therefore, efficient noninvasive calculation of FFR should be included in clinical routine.

Machine learning (ML) has been applied for accelerating the diagnostic differential process. By characterizing medical data and developing interpretations towards outcomes, ML could model the direct connection between the raw data and medical decision-making indices to relive the differential process from the massive data mining[8] and heavy computation expense.[9]^,[10] Previous studies showed that ML algorithm is capable to screen the lymph node metastases in breast cancer fifteen-fold faster than experience pathologist,[11] and to shorten the time-span for diffusion MRI data processing by twelve-fold faster.[12] CT-derived FFR in addition to image acquisitions may take 30 min to 4 h,[13] mainly due to time-consumed procedures of solving governing equations derived by discretization of the partial differential equations with large-scale elements in the arterial geometries,[14] The alternative approach could replace the procedures of solving governing equations by using ML algorithm to model the connections between complex representation of the coronary geometries and the subsequence variations of pressure distribution.

In this study, we presented a new deep learning approach, namely DEEPVESSEL-FFR, to model the connection between CT image and calculated FFR. The ML algorithm was constructed to interpret the physical properties of the blood pressure distribution in the patient-specific coronary arteries. Accordingly, we aim to evaluate the diagnostic performance of DEEPVESSEL-FFR in predicting the ischemic risk of the coronary arterial stenosis using ICA-FFR as a reference standard.

2. Method

2.1. Patients

This was a prospective, single-center, self-control study evaluating the diagnostic performance of coronary arterial ischemia assessment using DEEPVESSEL-FFR with ICA-FFR as reference standard. Data for evaluation of DEEPVESSEL-FFR were retrieved from Beijing Anzhen Hospital. The study was approved by the region ethic committees at the participating hospital, and patients provided written informed consent. DEEPVESSEL-FFR was conducted at core laboratory (Keya Medical technology, China). Evaluation of the diagnostic performance was performed in the blind manner to the ICA-FFR measurement.

Patients who underwent non-invasive CT with documented FFR within a maximum interval of 30 days between April 2017 and December 2017 were included following the criteria: (1) age between 18 to 75; (2) provided with inform consent; (3) CTA was performed in-hospital and fulfilling the quality check; (4) agreed with CTA-FFR test and ICA-FFR measurement; (5) at least one lesion (30% to 90% degree of stenosis) was presented in the coronary arterial branch with diameter larger than 2 mm; and (6) agreed to follow the designed protocol. Patients were excluded for (1) not adequate for ICA and FFR measurements; (2) previous coronary artery bypass grafting (CABG) or percutaneous coronary intervention (PCI) or other kinds of cardiac surgeries; (3) myocardial infections within 30 days before/after CTA, (4) tachycardia or significant arrhythmia; (5) body mass index (BMI) > 35 kg/m²; (6) acute symptoms; and (7) other factors that made the patient not adequate for the study. Drop out criteria was effective when (1) drop out request was made by enrolled patient; (2) severe adverse events; (3) CTA images failed quality check; and (4) other factors for the administrator decided that the patient was inadequate for the study.

2.2. Procedures

Images of CCTA were acquired with a 256-row, 16 cm detector CT system (Revolution CT, GE Healthcare). Beta-blockers were administered if necessary. ECG gating was implemented to acquire images at diastolic phase when contrast agent was fully perfused in the coronary arteries. FFR was performed following the clinical practice guideline.[15] FFR was measured in all cases using the VOLCANO instrument and a coronary pressure wire (PrimeWire PRESTIGE Plus pressure Guide Wire). After calibration and equalization, the pressure wire was advanced distally to the stenosis until the pressure sensor landed in a smooth coronary segment. Hyperemia was induced by using an intravenous infusion of adenosine (140 µg/kg per min). The pullback FFR data were recorded from the immediate downstream of the distal stenosis to the ostium of the coronary (PressureWire, St. Jude Medical). FFR was then calculated as the ratio between mean distal pressure (mPd) and mean aortic pressure (mPa). (Eq.1).

2.3. Evaluation of DEEPVESSEL-FFR

Evaluation of the DEEPVESSEL-FFR were performed by expert blind to the results of CCTA and ICA-FFR. DEEPVESSEL-FFR was performed online by using the pre-trained offline deep learning algorithm. The deep learning algorithm was trained on the basis of the data form previous work or clinical routines and the overall process were illustrated in Figure 1. Prediction of the DEEPVESSEL-FFR along the coronary arterial tree by a deep learning algorithm architecture constructed with multilevel neural network (MLNN) and bi-directional recursive neural network (BRNN), namely DBL-RNN. The network was arranged in two phases. The first phase was to interpret feature from input by using MLNN with three fully connected layers.[10] Input were extracted from the image-based reconstruction of arterial tree as descripted,[16] including lesion characteristics and proximal/distal markers subsequently defined for each lesion. The fully connected layers transformed input characteristics from each note along the arterial tree into feature with weight Vs for the next phase. The second phase comprised of BRNN that receive the features sequence to the next layer. Recurrent connections that allow inputing historical information in a chain of data sequence both forward and backward passing the surrounding notes by applying weights during the training process. The output layers returned DEEPVESSEL-FFR value correspondingly to each input neuron. During the training, ground truths were applied to the output layers, which were generated by solving the Navier-Stokes equations with ICA-FFR as a reference. The Stochastic gradient-descent algorithm was applied for optimization of weight vector until minimization of the error between ground truths and predictions convergence was achieved.

Figure 1. The process of DEEPVESSEL-FFR.

MLNN: multilevel neural network; RNN: multilevel neural network.

2.4. Implementation

The deep learning algorithm was trained offline in the core lab in Keya medical technology Inc. (Shenzhen, China) and distributed as an online DEEPVESSEL-FFR platform. The architecture is further detailed in the Online Appendix. The CCTA image data in DICOM (Digital Imaging and Communications in Medicine) format were transferred to the platform and DEEPVESSEL-FFR distribution in the coronary artery tree was returned.

2.5. Statistical analysis

Baseline characteristics are presented as n (%) for categorical variable anatomic indices. Qualitative variables of the baseline characteristics were examined by using t-test and the quantitative ones were examined using Pearson's Chi-squared test or Fisher's test. Two tailed P < 0.05 was considered statistically significant. To evaluate the diagnostic performance of DEEPVESSEL-FFR, the primary evaluation index was the area under receiver operation curve (AUC) and the secondary index included accuracy, sensitivity, specificity, positive predicted value (PPV) and negative predictive value (NPV) in per-patient based and per-vessel based manner, respectively. Statistic calculation was performed using SPSS version 19.0 and MedCalc software (Version 15.6.1).

3. Results

3.1. Patient demographics

The study population comprised 68 patients, five patients were excluded based on the excluding criteria, finally 63 patients with 71 vessels (left anterior descending artery: 45.1%, 32 vessels; left circumflex artery: 28.4%, 21 vessels; and right coronary artery: 25.4%, 18 vessels) were included. Baseline characteristics of the study population were illustrated in Table 1. No significant differences in baseline characteristic were found in inter-center comparison, including sex [χ² = 0.156, P = 0.925 (> 0.05)], age [F = 0.015, P = 0.985 (> 0.05)], height [F = 1.162, P = 0.316 (> 0.05)], weight [F = 1.087, P = 0.340 (> 0.05]], and BMI [F = 0.197, P = 0.821 (> 0.05)]. Abnormal FFR (≤ 0.80) was observed in 35 vessels (55.6%).

Table 1. Baseline characteristics.

Male	32 (50.8%)
Age, yrs	68.8 ± 8.63
BMI, kg/m2	25.3 ± 3.4
Diabetes mellitus	31 (49.2%)
Hypertension	33 (52.4%)
Hyperlipidemia	27 (42.8%)
Smoking	27 (42.8%)
LAD	32 (45.1%)
LCX	21 (28.4%)
RCA	18 (25.4%)
FFR ≤ 0.80	35 (55.6%)

Data were presented as n (%) or mean ± SD. BMI: body mass index; FFR: fractional flow reserve; LAD: left anterior descending artery; LCX: left circumflex artery; RCA: right coronary artery.

3.2. Correlation and agreement between FFR and DEEPVESSEL-FFR

DEEPVESSEL-FFR showed good correlation to the invasive FFR measurement [R = 0.686 (0.567, 0.799), P < 0.00001]. A good agreement was found between DEEPVESSEL-FFR and FFR [–0.006 ± 0.091 (95% CI: –0.1943 to 0.1816), P = 0.619] on a per-patient level analysis (Figure 3). DEEPVESSEL-FFR showed good correlation to the invasive FFR measurement [R = 0.683 (0.570, 0.792), P < 0.00001]. A good agreement was found between DEEPVESSEL-FFR and FFR [–0.005 ± 0.086 (95% CI: –0.1943 to 0.1816), P = 0.652] on a per-vessel level analysis (Figure 3).

Figure 3. Correlation (A & C) and agreement (B & D) analysis on a per-patient level and per-vessel level.

DVFFR: DEEPVESSEL fractional flow reserve.

Figure 2. Representative examples of subjects from the study.

(A): Multiplanar reformat of coronary computed tomography (CT) angiogram; (B): the left anterior descending artery (blue arrow) and DEEPVESSEL-FFR 0.81; (C): invasive coronary angiogram; and (D): invasive FFR measurement, a measured FFR value of 0.81. FFR: fractional flow reserve.

3.3. Accuracy of DEEPVESSEL-FFR for diagnosis of ischemia-related lesions

The cut-off value referring to the FFR measurement was taken as ≤ 0.8. In comparison to the CTA based severity of stenosis, DEEPVESSEL-FFR improved diagnostic performance in determining ischemia-related lesions with AUC of 0.664 (95% CI: 0.524–0.772) vs. 0.928 (95% CI: 0.833– 0.978) and 0.662 (95% CI: 0.540–0.770) vs. 0.933 (95% CI: 0.848–0.979) for patient-based and vessel-based evaluations (Figure 4), respectively. DEEPVESSEL-FFR with per-patient accuracy, sensitivity, specificity, positive predictive value, and negative predictive value was 87.30% (95% CI: 76.50%–94.35%), 97.14% (95% CI: 75.00%–82.93%), 95.45% (77.16%–99.88%), respectively. Per-vessel accuracy, sensitivity, specificity, positive predictive value, and negative predictive value was 88.73%, 97.56%, 76.67%, 85.11%, 95.83%, respectively. Details were illustrated in Table 2.

Figure 4. AUC of DEEPVESSEL-FFR vs. coronary CTA for demonstration of ischemia (FFR 0.80) on a per-patient and per-vessel basis.

AUC: area under receiver-operation characteristics curve; CTA: computed tomography angiography; FFR: fractional flow reserve.

Table 2. Diagnostic performance of DEEPVESSEL-FFR in per-patient and per-vessel level.

	Per-patient	95% CI	Per-vessel	95% CI
Accuracy	87.30%	76.50%–94.35%	88.73%	78.99%–95.01%
Sensitivity	97.14%	85.08%–99.93%	97.56%	87.14%–99.94%
Specificity	75.00%	53.13%–89.31%	76.67%	57.72%–90.07%
PPV	82.93%	67.94%–92.85%	85.11%	71.69%–93.80%
NPV	95.45%	77.16%–99.88%	95.83%	78.88%–99.89%

FFR: fractional flow reserve; NPV: negative predictive; PPV: positive predicted value.

3.4. Computational performance of DEEPVESSEL-FFR.

Evaluation of DEEPVESSEL-FFR was conducted online by transferring the raw CCTA data to the DEEPVESSEL-FFR platform and the result of prediction was provided by the pre-trained deep learning algorithm. The platform was distributed in the server located in the core laboratory of Keya medical technology Inc. co., the average computation time required for computing DEEPVESSEL-FFR in the entire coronary tree for each patient in present study was 120 ± 13 s regardless of the device configuration for CCTA DICOM data storage.

4. Discussion

The CFD approach has been frequently applied to compute the FFR from CTA. This technique heavily relied on the quality of the underlying computational models and sophisticated boundary conditions and required a few hours for computation.[13]^,[17] We developed the DEEPVESSEL-FFR platform using the emerging deep learning technique to calculate the FFR value from CTA images in 5 min. The computational time of our platform for calculating the FFR value for a new case was 120 ± 13 s regardless of the configuration of the device for CCTA DICOM data storage, which is much faster than that of existing CFD models on a moderate workstation.

We demonstrated an efficient workflow for non-invasive functional assessment method for ischemic-risk of the stenotic coronary arteries. On one hand, the raw CCTA images in DICOM format of each patient was upload to the DEEPVESSEL-FFR platform without conventional procedures, including pre-processing for segmentation of the arterial regions and extraction of the input features and post-processing for extraction of diagnostic index from calculations. Therefore, significant amount of time-span was spared. On the other hand, the online nature of the platform had eliminated the time expense variation associating with configuration of the local processer device. In additional to the fast prediction manner of deep learning algorithm, DEEPVESSEL-FFR could be clinically suitable solution for CAD screening.

A Meta-analysis included 908 vessels from 536 patients in five studies was performed by Cook, et al.[18] The overall per-vessel diagnostic accuracy of FFR-CT was 81.9% (95% CI, 79.4%–84.4%). The overall per-vessel diagnostic accuracy of DEEPVESSEL-FFR was over 88.73%, which is similar to the previous published method of computational FFR. We also observed the improvement of the DEEPVESSEL-FFR in detecting ischemic-risk compared to using CTA alone.

4.1. Limitations

A relatively small number of patients were included which may incur a selection bias. Despite the small sample size most pathological FFR measurements were about LAD, potentially lead to confined findings. Further studies with larger sample sizes are needed to demonstrate the diagnostic accuracy of our findings.

4.2. Conclusion

The DEEPVESSEL-FFR platform achieved satisfactory diagnostic accuracy in detecting ischemia when FFR served as the gold standard. The performance of this platform has demonstrated its potential in the clinical routine.