Comparison of diagnostic performance between quantitative flow ratio, non-hyperemic pressure indices and fractional flow reserve

Abstract

Background

Quantitative flow ratio (QFR) is an estimate of fractional flow reserve (FFR) and is derived from 3-dimensional quantitative coronary angiography. The DILEMMA score is an angiographic technique developed to predict FFR. Unlike other diastolic indices such as instantaneous wave-free ratio (iFR), diastolic pressure ratio (dPR) and dPR_25–75, neither QFR nor DILEMMA score require pressure wires. This study sought to compare the diagnostic performance of QFR, diastolic indices and DILEMMA score to predict FFR.

Methods

Between January 2010 and December 2013, patients who underwent invasive coronary angiography and FFR assessments were retrospectively studied. iFR and dPR were derived from FFR pressure tracings. QFR was computed using commercial software.

Results

Eighty-five lesions (25% FFR significant) were included in this study. Median FFR was 0.88 (0.81–0.92). QFR (r_s=0.801), iFR (r_s=0.710), dPR (r_s=0.716), dPR_25–75 (r_s=0.715) and DILEMMA score (r_s=−0.623) significantly correlated with FFR (P<0.001). QFR ≤0.8 had a specificity, sensitivity, positive predictive value (PPV) and negative predictive value (NPV) of 95%, 86%, 86% and 95% respectively of predicting significant FFR (P<0.001). Receiver-operating characteristic (ROC) analysis revealed the AUC to predict significant FFR for QFR (0.947), iFR (0.880), dPR (0.883), dPR_25–75 (0.880) and DILEMMA score (0.916) were not significantly different. However, QFR was a better predictor of FFR than iFR (0.947 vs. 0.770, P<0.01).

Conclusions

QFR had excellent correlation and accuracy as measured against FFR. When compared to other diastolic indices and DILEMMA score, QFR performed at least as well as the other indices. QFR predicts FFR better than it predicts iFR. QFR is a convenient tool to assess significance of coronary stenosis and a reliable alternative to pressure-wire based indices. Prospective studies are required to investigate the performance and cost-effectiveness of QFR when independently used to guide clinical decision making.

Introduction

Treatment of physiologically significant coronary artery stenoses is associated with improvement in symptoms and cardiovascular outcomes (1-5). Methods to determine the physiological significance of stenoses have evolved over the past two decades. Fractional flow reserve (FFR) is currently the ‘gold standard’ to assess the significance of coronary stenoses (6). FFR-guided percutaneous coronary intervention (PCI) is superior to visual angiography-guided PCI and results in better patient outcomes (7,8). Although its use has increased in the developed world, FFR remains under-utilized due to the additional time taken and costs incurred to interrogate stenoses with a pressure wire, limitations associated with induction of hyperaemia, and ongoing operator confidence in their visual assessment of lesion severity (9-12).

Instantaneous wave-free ratio (iFR) is another physiological index to assess functional significance of coronary stenoses. In contrast to FFR, iFR is derived by calculating the mean ratio of the pressure distal to a lesion (Pd) over the aortic pressure (Pa) during the wave-free period in diastole, circumventing the need to pharmacologically induce hyperaemia (13). Like FFR, iFR requires the use of a pressure wire. Recent studies have shown comparable results between iFR and FFR, with an iFR threshold of 0.89 comparable to an FFR threshold of 0.80, to suggest physiological significance (14-17). Van’t Veer and colleagues recently demonstrated that all diastolic resting indices were identical to iFR, both numerically and with respect to their agreement with FFR (18). While the dPR (diastolic pressure ratio) calculates the mean Pd/Pa over the entire diastolic period, dPR_25–75 only considers between 25% and 75% of the diastolic period (18).

As coronary angiography remains the cornerstone in clinical decision making, development of an angiography-based functional assessment of coronary artery stenosis would be ideal. This would overcome the growth limitations of FFR and iFR attributed to expenses associated with pressure wires. Two such methods are the DILEMMA score and quantitative flow ratio (QFR). The DILEMMA score evaluates coronary artery lesions by considering the degree of luminal stenosis, length of the lesion and area of myocardium at jeopardy subtended by that lesion (19). Our group had evaluated its diagnostic accuracy against FFR in recent studies (19,20). QFR is a novel method which utilises advancements in mathematical modelling techniques and scientific understanding of principles of computational fluid dynamics, to derive an angiography based assessment of functional significance (21). The diagnostic accuracy of QFR in identifying haemodynamically significant coronary artery stenoses had recently been demonstrated (21).

The aim of this study is to compare the diagnostic accuracy of these various novel contemporary techniques against the gold standard FFR, in stable coronary artery disease.

Methods

Patients

We retrospectively examined patients with stable coronary artery disease who underwent elective coronary angiography and FFR assessment at a major tertiary referral hospital (Monash Medical Centre) in Melbourne, Australia between January 2010 and December 2013. These dates were selected as angiography studies in our institution within these dates had a frame acquisition rate of 15 frames per second (required to perform QFR analyses). Data acquisition for this retrospective study was previously approved by our institutional human research ethics committee (Approval ID: 10326). All lesions had diameter stenosis between 30–90% and a reference vessel diameter ≥2 mm according to visual assessment. FFR was clinically indicated and performed at the discretion of their treating cardiologist. At our institution, and consistent with conventional methods, the clinically acceptable drift on FFR acquisition was <0.02. We identified patients who had retrievable pressure wire tracing data. Exclusion criteria included patients with bypass grafts, culprit vessels that collateralize other vessels, culprit vessels in the setting of myocardial infarction, lesions in diagonal vessels, vessels with myocardial bridging and those where angiographic acquisition image quality did not allow for QFR analysis.

Invasive coronary angiography, FFR and QCA

Invasive coronary angiography and FFR were performed as per standard practice via the femoral or radial approach. An FFR value ≤0.80 was taken to define functionally significant stenoses. The pressure wire (Pressure wire Certus 6, St Jude Medical, USA) was calibrated and electronically equalised with the aortic pressure before being placed in the distal third of the coronary artery being interrogated. Intracoronary glyceryl trinitrate (100 mcg) was injected to minimise vasospasm. Intravenous adenosine was administered (140 mcg/kg/min) through an intravenous line in the antecubital fossa. At steady-state hyperaemia, FFR was recorded on a RadiAnalyzer Xpress (St Jude Medical Systems, USA), calculated by dividing the coronary pressure measured with the sensor placed distal to the stenosis (Pd) by the aortic pressure measured through the guide catheter (Pa). Quantitative coronary angiographic analysis was performed to derive minimal luminal diameter (MLD) and lesion length.

iFR and dPR

The iFR calculated was similar, but not identical, to that described originally (13). Briefly, we analysed our diastolic period from the dicrotic notch up until 10 ms before the end of the diastolic period, rather than 5 ms. This was because pressure tracing data were available every 10 ms rather than 5 ms.

The dPRs we selected were dPR and dPR_25–75. While dPR measured the average Pd/Pa ratio over the entire diastolic period, dPR_25–75 measured the average Pd/Pa between 25% and 75% of the period into diastole (18). This period was identified by manually selecting values between the dicrotic notch and end of diastole.

All calculations were derived after averaging values over five cardiac cycles.

QFR

The QFR analysis was performed using a validated software (QAngio XA3D 3.1.1; Medis Medical Imaging System, Leiden, The Netherlands) by two independent investigators who were unaware of the FFR/iFR/dPR/dPR_25–75/DILEMMA results. Two angiographic image projections acquired at ≥25º apart that presented the least foreshortening of the stenosis and minimum overlap of the main vessel and side branches were selected for analysis. All angiographic images were acquired at 15 frames per second. The lumen contour was automatically delineated and manual correction performed as appropriate. A 3D anatomical vessel model without side branches was derived from the software and QFR computation was performed using a specific flow model: contrast-flow QFR (cQFR) previously described by Tu et al. (21). In brief, cQFR was computed using a modelled hyperemic flow velocity on the basis of the thrombolysis in myocardial infarction (TIMI) frame count analysis without pharmacologically-induced hyperemia. The TIMI frame count analysis was carried out on either one of two angiographic projections that provided a more well-defined contrast flow. The QFR value was obtained at the distal coronary artery corresponding to the site of the FFR measurement. Diagonal lesions were excluded due to equivocal classification as there were disparate methods to compute QFR on LAD and non-LAD vessels.

DILEMMA score calculation

Methodology to calculate the DILEMMA Score is described in an earlier study by our institution, and summarised in Table 1 (19). Briefly, the DILEMMA score was a summation of points attributed to a target lesion based on three characteristics: minimum luminal diameter (maximum 4 points), lesion length (maximum 3 points) and myocardium at risk (maximum 5 points). The area of myocardium at risk was calculated from angiography using the BARI myocardial jeopardy index, which has been validated with cardiac MRI (22). Each lesion is assigned a DILEMMA score of between 0 and 12.

Table 1. Calculation of the DILEMMA score.

Variable	Value	Points
MLD (mm)	<1.1	4
	1.1–1.5	1
	>1.5	0
Lesion length (mm)	>18	3
	9–18	1
	<9	0
Jeopardy index (%)	>35	5
	18–35	1
	<18	0
Total		Maximum =12

Statistical analysis

Statistical analyses were performed with IBM SPSS Statistics 25.0 (IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) and R programming language version 3.6.2 (RStudio for Windows, Version 1.2.5033). Variables were tested for normality with visual assessment, the Shapiro-Wilk test and a test of skewness.

Wilcoxon paired signed-rank test was performed as a non-parametric substitute for paired t-test analysis. Correlation between non-parametric continuous variables were evaluated with Spearman’s rank correlation coefficient. Statistical comparison of coefficients was performed after Fisher-Z transformation. For all analyses, a two-tailed P value of less than 0.05 was considered statistically significant.

A receiver-operating characteristic curve was established with reference FFR ≤0.80 or iFR ≤0.89 to correspond to physiologically significant obstructive coronary artery stenosis and discriminatory function assessed for statistical significance.

Owing to the repeated-measures nature of the study, a general estimating equation approach was used assuming a binomial probability distribution.

Results

Clinical characteristics

We identified 115 lesions from 54 patients, for which FFR were performed between January 2010 and December 2013, with a radiographic frame rate of 15 fps, the minimum required for QFR analysis (Table 2). We excluded 15 cases due to poor image acquisition or suboptimal raw data capture of pressure tracings, 13 cases due to the lesion involving the diagonal artery and two cases due to presence of muscle bridging (Figure 1). There were five cases for which the pressure tracing did not have a sufficient time period of recording in the pre-hyperaemic phase and therefore an iFR, dPR or dPR_25–75 were not able to be calculated accurately.

Table 2. Characteristics of study population.

Characteristics	Number (n=47)
Age (y)	65.1±10.1
Male (n, %)	29 (61.7)
Risk factors (n, %)
Diabetes	11 (23.4)
Hypertension	27 (57.4)
Hypercholesterolaemia	31 (66.0)
Smokers	22 (46.8)
Family history of IHD	20 (42.6)
Previous MI	7 (14.9)
Previous PCI	10 (21.3)

IHD, ischemic heart disease; MI, myocardial infarction; PCI, percutaneous coronary intervention.

Figure 1.

Flow diagram of patient enrolment into study.

Among the 85 lesions included in the analysis, 38 were from the LAD (45%), 25 were from the LCX or branches (29%) and 22 were from the RCA or branches (26%). There were 21 vessels (25%) with FFR ≤0.80. The mean FFR was 0.86±0.09 and the median FFR was 0.88 (0.81–0.92) (Table 3).

Table 3. Characteristics of included vessels.

Characteristics	Number (n=85)
Vessel with FFR ≤0.8, n (%)	21 (24.7)
Lesion length, mm	11.9±10.2
Diameter stenosis, %	35.3±14.7
No. of vessels interrogated by FFR per patient, n (%)
1	22 (46.8)
2	12 (25.6)
≥3	13 (27.7)
Lesion location, n (%)
Left anterior descending artery	38 (44.7)
Left circumflex artery	15 (17.6)
Obtuse marginal branch	10 (11.8)
Right coronary artery	17 (20.0)
Posterior descending artery	4 (4.7)
Posterolateral branch	1 (1.2)
Functional assessment indices (median; 25th–75th percentile)
Fractional flow reserve	0.88 (0.81–0.92)
Instantaneous wave-free ratio	0.92 (0.84–0.97)
Diastolic pressure ratio (complete diastole)	0.92 (0.85–0.97)
Diastolic pressure ratio (25–75% of diastole)	0.91 (0.84–0.97)
Quantitative wave free ratio	0.89 (0.81–0.95)
DILEMMA score	2 [1–6]

Correlation between FFR, QFR, iFR, dPR, dPR_25–75 and DILEMMA score

Using the Spearman’s rank correlation coefficient (r_s), FFR was significantly and positively correlated to QFR (r_s=0.801; P<0.001), iFR (r_s=0.710; P<0.001), dPR (r_s=0.716; P<0.001) and dPR_25–75 (r_s=0.715; P<0.001). There was statistically significant negative correlation between FFR and the DILEMMA score (r_s=−0.623; P<0.001). Direct comparison of correlations between FFR against QFR and FFR against iFR did not show statistical significance (P=0.17).

There was excellent correlation between the diastolic indices: iFR and dPR (r_s=0.993; P<0.001); iFR and dPR_25–75 (r_s=0.995; P<0.001).

The correlation between FFR and QFR was greater than the correlation between iFR and QFR (r_s=0.801 vs. r_s=0.516, P<0.001). Scatter plots can be seen in Figure 2.

Figure 2.

Scatter plots of FFR against various indices. (A) FFR against QFR; (B) FFR against iFR; (C) FFR against dPR; (D) FFR against dPR_25–75; (E) FFR against DILEMMA score; (F) iFR against dPR and dPR_25–75. Each circle represents a case and size of circle (small, medium, large) corresponds to number of cases. (A,B) Orange circles represent correctly classified lesions and blue circles represent incorrectly classified lesions. FFR, fractional flow reserve; QFR, quantitative flow ratio; iFR, instantaneous wave-free ratio; dPR, diastolic pressure ratio.

Diagnostic performance of iFR, dPR and dPR_25–75 to predict FFR ≤0.8

There were 30 (38%) vessels with iFR ≤0.89. Mean iFR was 0.91±0.08 and median iFR was 0.92 (0.84–0.97). The specificity, sensitivity, positive predictive value (PPV) and negative predictive value (NPV) of iFR ≤0.89 to predict FFR ≤0.8 were 79%, 89%, 57% and 96% respectively. The overall diagnostic accuracy was 81%.

There were 30 (38%) vessels with dPR ≤0.89. Mean dPR was 0.90±0.08 and median dPR was 0.92 (0.85–0.97). The specificity, sensitivity, PPV and NPV of dPR ≤0.89 to predict FFR ≤0.8 were 80%, 89%, 59% and 96% respectively. The overall diagnostic accuracy was 83%.

There were 31 (39%) vessels with dPR_25–75 ≤0.89. Mean dPR_25–75 was 0.90±0.09 and median dPR_25–75 was 0.91 (0.84–0.97). The specificity, sensitivity, PPV and NPV of dPR_25–75 ≤0.89 to predict FFR ≤0.8 were 77%, 89%, 55% and 96% respectively. The overall diagnostic accuracy was 80%.

Diagnostic performance of QFR to predict FFR ≤0.8 and iFR ≤0.89

There were 21 (25%) vessels with QFR <0.8. Mean QFR was 0.86±0.13 and median QFR was 0.89 (0.81–0.95). A Wilcoxon paired signed-rank test had a P value of 0.357, suggesting no statistically significant difference between individual QFR and FFR values. The specificity, sensitivity, PPV and NPV of QFR <0.8 to predict FFR ≤0.8 was 95%, 86%, 86% and 95% respectively. The overall diagnostic accuracy was 93%.

When iFR <0.89 was used as the reference standard, the specificity, sensitivity, PPV and NPV of QFR was 94%, 53%, 84% and 77%. The overall diagnostic accuracy was 79%.

Diagnostic performance of DILEMMA score to predict FFR ≤0.8

A DILEMMA score of ≤2 (n=43) had an NPV of 95% for identifying FFR negative lesions while DILEMMA score of ≥9 (n=3) had a PPV of 100% to identify FFR positive lesions.

Comparison of receiver operating characteristic analyses

A receiver-operating characteristic (ROC) curve was created to assess the accuracy of QFR, iFR, dPR, dPR_25–75 and DILEMMA score (Figure 3). The discriminatory function was numerically greatest for QFR (AUC =0.947; P<0.001), followed by DILEMMA (AUC =0.916; P<0.001), dPR (AUC =0.883; P<0.001), iFR (AUC =0.880; P<0.001) and dPR_25–75 (AUC =0.880; P<0.001). Differences in AUC between the indices were not statistically significant (P>0.05).

Figure 3.

ROC curve for QFR, iFR, dPR, dPR_25–75 and DILEMMA tested against an FFR <0.80. ROC, receiver-operating characteristic; QFR, quantitative flow ratio; iFR, instantaneous wave-free ratio; dPR, diastolic pressure ratio; FFR, fractional flow reserve.

The discriminatory function of QFR was lower when iFR ≤0.89 was used as the reference standard instead of FFR ≤0.80 (AUC =0.770 versus 0.947; P<0.01).

Discussion

In this study, we compared the diagnostic accuracy of novel contemporary methods (iFR, dPR, dPR_25–75, QFR and DILEMMA score) to determine functional significance of coronary artery stenosis against the reference standard, FFR. We showed that QFR compared favourably to iFR, dPR, dPR_25–75 and DILEMMA score to predict significant FFR. We showed that QFR and DILEMMA scores can be potential angiography based “gatekeepers” to FFR.

iFR, dPR, dPR_25–75 versus FFR

The overall diagnostic accuracy of iFR (81%), dPR (83%) and dPR_25–75 (80%) to predict significant FFR in our study is comparable to previous studies. Prospective studies by Escaned et al. and Pisters et al. showed that iFR with a cut-off of 0.89 had an accuracy of 82.5% and 82% respectively (23,24). In our study, the Spearman’s Rho correlation between iFR and dPR was 0.993 and iFR and dPR_25–75 was 0.995. This was similar to 0.993 and 0.997 respectively as obtained by Van’t Veer et al. (18).

iFR, dPR and dPR_25–75 versus QFR

To the best of our knowledge, the present study is the first to compare QFR with iFR, dPR and dPR_25–75. A recent study by Emori et al. found that QFR had superior diagnostic accuracy to iFR (94% vs. 74%) in predicting lesions with FFR less than or equal to 0.80 (25). Additionally, similar to our findings, Hwang et al. found QFR had better discriminatory function against a reference standard of FFR rather than iFR (0.953 vs. 0.880) (26).

DILEMMA score versus FFR

An alternative method requiring no additional software or expense to angiography, was the DILEMMA score. In our study, we showed that a DILEMMA score of ≤2 and ≥9 have high NPV and PPV respectively, consistent with previous findings which show that DILEMMA score of ≤2 had a NPV of 96.3% and a score ≥9 had a PPV of 88.9% (20).

QFR versus FFR

QFR is an emerging tool solely based on angiography. It relies on complex mathematical methods built upon principles of computational fluid dynamics. In contrast to FFR and iFR, QFR neither requires an intracoronary pressure wire, nor does it need adenosine administration to induce hyperaemia. It can be performed almost immediately with a result derived within minutes after angiography has completed. QFR has demonstrated substantial reproducibility with excellent inter-observer agreement (27). Our study is consistent with others that have investigated QFR. The international multicentre FAVOR Pilot study by Tu et al. compared QFR against the gold standard FFR and had similar results with a statistically significant Pearson’s correlation of 0.77, an AUC of 0.92 and an overall diagnostic accuracy of 86% (21). Given the advantages highlighted and speed with which QFR can be derived, QFR could play a potential role as a gatekeeper to FFR. Studies that included a hybrid approach with QFR and FFR suggest that in approximately two-thirds of cases, a pressure-wire could be avoided whilst maintaining a diagnostic accuracy of 95% (28,29).

Assessment of QFR as a diagnostic test

The prevalence of disease in a population affects the PPV and NPV of a diagnostic test in question. In our study, the proportion of lesions that were significant on FFR assessment were 25%. In the FAME study, the proportion of lesions that were angiographically of intermediate severity and later found to be FFR significant were 35% (7). Thus, given our lower prevalence rate of FFR significance, our results may overestimate the NPV and underestimate the PPV of QFR as a diagnostic test.

Clinical implications

It is commonly accepted that visual or 2D-QCA estimations of coronary stenoses are often inaccurate and not reflective of a stenotic lesion’s physiological significance. However, the process of deriving pressure indices is not without clinical risk, upfront financial cost for the institution, and added procedural time. Even though studies such as RIPCORD (30), R3F (31), and POST-IT (32) show that routine use of FFR changes revascularization strategy in 26–44% of cases, the adoption of FFR is not widespread internationally. Landmark studies demonstrate that FFR guidance typically lowers the number of lesions that require PCI, reduces the overall hospital care costs and improves patient outcomes (8,33). Additionally, FFR-guided treatment seems to lead to an overall economic benefit when compared to angiographically guided management (34-36). However, an overall economic benefit may not necessarily reflect a better financial position for the institute undertaking the procedure.

In today’s environment, where procedures need to pass the scrutiny of financial impact, clinical safety and clinical value, there may be a role for QFR. QFR may lower an institution’s financial burden by substantially reducing the number of lesions that proceed to FFR analysis, without compromising accuracy. As the input data necessary for QFR is the angiogram itself, the need to proceed to FFR can quickly be determined by the end of the initial image acquisition. This would provide meaningful information to guide the cardiologist and is important as the adoption of FFR, while increasing, is not universal. The cost of this would include training time of the technician involved in the analysis, as well as licensing for necessary software.

The DILEMMA score aims to provide guidance to the cardiologist without added cost to the process of angiography. While our study demonstrates that it can be clinically effective, the uptake of the DILEMMA score is also quite low, perhaps due to the tedious and arguably lengthy nature of the necessary calculations.

Limitations on the application of QFR are the circumstances and conditions within which QFR can be applied confidently. Situations where it may not be appropriate to use QFR to estimate FFR include extensive image foreshortening and overlap, ostial LCx lesions, left main stenosis and various others outlined by the software vendor. Other conditions where reliability and interpretation need further investigation are those that are also uncertain in FFR, for instance the presence of severe aortic stenosis.

QFR and the DILEMMA score are both angiographically-derived methods which, upon comparison, have their individual strengths. The DILEMMA score incorporates QCA derived parameters such as MLD and lesion length as well as an estimate of myocardial mass subtended by the stenosis using the BARI myocardial jeopardy index (19). Although the DILEMMA score has been shown to reduce the number of patients requiring invasive physiological assessment in a large multi-centre study of patients with intermediate coronary artery stenoses (20), the DILEMMA score does not directly estimate the FFR. QFR, on the other hand, provides an estimate of the FFR. However, it requires proprietary software and the added costs that come with that.

Study limitations

Our results represent a retrospective small, single-centre experience and hence needs confirmation with larger multi-centre studies. The major barrier to the number of patients included in our study was the availability of retrievable pressure tracing data between 2010 and 2013. Additionally, as our centre had transitioned to a lower frame rate as standard protocol to minimise radiation for angiography, we were unable to assess more recent angiographic cases in order to meet the frame rate requirement (>12.5 frames per second) to perform QFR analysis. Future studies to assess the utility of contrast-QFR on studies acquired at lower frame rate are therefore required. Side branches of bifurcation lesions with Medina type 1,1,1 or 1,0,1 were not assessed as generalizability of QFR to the side branches of bifurcation lesions have not been thoroughly assessed.

Conclusions

QFR had excellent correlation and diagnostic performance as measured against FFR when compared to iFR, dPR, dPR_25–75 and DILEMMA score. QFR performed superiorly against a reference standard of FFR compared to iFR. QFR is a convenient tool to assess the physiological significance of coronary stenoses, providing a reliable alternative to other pressure wire based indices. Further studies are required to assess QFR performance in a prospective manner, particularly with outcome studies. Additionally, cost effectiveness studies are necessary for adoption into routine clinical practice.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Data acquisition for this retrospective study was previously approved by our institutional human research ethics committee (Approval ID: 10326). Because of the retrospective nature of the research, the requirement for informed consent was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.