Summary
Objective
Since visual estimation of the extent of vessel stenosis may vary between operators, we aimed in this study to investigate both inter-observer variability and consistency between the estimation of an operator and quantitative coronary analysis (QCA) measurements.
Methods
A total of 147 elective percutaneous coronary intervention patients with 155 lesions between them were consecutively enrolled in the study. These patients were evaluated for visual estimation of lesion severity by three operators. The lesions were also evaluated with QCA by an operator who was blinded to the visual assessments. Reference diameter, minimal lumen diameter, percentage diameter of stenosis, percentage area of stenosis and diameter of lesion length from the proximal lesion-free segment to the distal lesion-free segment were calculated using a computerised QCA software program.
Results
There was a moderate degree of concordance in the categories 70–89% (kappa: 0.406) and 90–99% (κ: 0.5813), whereas in the categories < 50% and 50–69% there was a low degree of concordance between the visual operators (κ: 0.323 and κ: 0.261, respectively). There was a low to moderate grade of concordance between visual estimation and percentage area of stenosis by QCA (κ: 0.30) but there was no concordance between visual estimation and percentage diameter of stenosis by QCA (κ: –0.061). Also, there was a statistically significant difference between QCA parameters of percentage diameter of stenosis and percentage area of stenosis (58.4 ± 14.5 vs 80.6 ± 11.2 %, p ± 0.001).
Conclusion
Visual estimation may overestimate a coronary lesion and may lead to unnecessary coronary intervention. There was low concordance in the categories < 50% and 50–69% between the visual operators. Percentage area of stenosis by QCA had a low to moderate grade of concordance with visual estimation. Percentage area of stenosis by QCA more closely reflected the visual estimation of lesion severity than percentage diameter of stenosis.
Keywords: coronary stenosis, quantitative coronary analysis, coronary angiography
Standard coronary angiography is the gold standard in the diagnosis of coronary artery disease. Most laboratories use visual estimation to predict the severity of coronary lesions. Many patients undergo coronary revascularisation according to visual estimation of their coronary stenosis. Unfortunately, visual estimation may vary between operators.
In 1971, Gensini et al. first introduced a new electronic measurement system by drawing the vessel contour with a cursor.1 From the early 80s, many computer-based quantitative coronary assessment (QCA) programs have been developed and embedded in angiographic devices. Nowadays, modern QCA programs enable more accurate assessment and more reproducible measurement of coronary stenosis in an operatorindependent way.
Many studies have shown inter-operator variation and discrepancy between visual estimation and QCA analysis. Most of these studies were performed before 2000.2-8 In a recent study performed by Nallamothu et al., the authors found that many operators tend to estimate coronary lesions more severely than QCA measurement.9 This is consistent with older studies. In the light of this study, many patients who did not have severe lesions according to QCA have undergone unnecessary revascularisation procedures based on visual estimation.
In this retrospective study, we aimed to investigate both inter-observer variability and consistency between the visual estimation of a primary operator and QCA measurement in a blinded manner in patients who had had elective percutaneous coronary intervention (PCI) in our clinic.
Methods
A total of 147 consecutive patients who had had elective PCI between January and June 2015 were enrolled in the study. We obtained the data for these patients from the records of our catheterisation laboratory. Patients who had had acute myocardial infarction and totally occluded coronary lesions were excluded from the study.
A total of 147 patients with 155 lesions between them were identified and retrospectively enrolled in the study in a consecutive manner. These patients’ records were evaluated for visual estimation of their lesion severity by two other operators who were blinded to the previous primary operator’s visual estimation. We also categorised the lesions as percentages according to their severity: < 50, 50–69, 70–89 and 90–99%. Three visual estimations (qualitative evaluation) were therefore obtained for each lesion.
For QCA analysis, first, the lesion was evaluated in multiple views for quality of the images, excessive foreshortening, sidebranch overlap and severity of stenosis. The frame demonstrating the most severe narrowing with the best image quality and least foreshortening was selected in end-diastole and then calibration was done using the tip of the catheter. Disease-free segments of proximal and distal coronary segments were used as reference segments.
Thereafter, the software automatically detected the contour after manually tracing a central line through the lesion. The proximal and distal coronary segments should be relatively free of disease and were referred to as the reference diameter. Vessel contour was automatically detected by the software and edge detection was corrected if necessary. In cases of multi-lesion intervention, each lesion was evaluated separately (Fig. 1).
Fig. 1.
Quantitative coronary analysis of a lesion in the left circumflex coronary artery
Complete QCA analysis of the lesions of each patient was performed by another operator who was blinded to the visual assessment of the lesions. Reference diameter (the diameter of the disease-free segments of the proximal and distal vessels), minimal lumen diameter, percentage of stenosis, percentage area of stenosis and lesion length from the proximal lesionfree segment to the distal lesion-free segment in diameter were calculated using a computerised QCA software program (Axiom Artis Zee, Siemens, Germany). One QCA (quantitative evaluation) measurement was thus obtained for each lesion.
Statistical analysis
Continuous variables are expressed as mean ± SD and categorical variables as numbers and percentages. All data were evaluated by IBM SPSS (Statistical Package for Social Sciences, version 22). Kappa analysis was used for evaluation for concordance of visual assessments between operators. The difference between visual assessment and QCA was determined using the paired Student’s t-test. Concordance between visual assessment and QCA was tested with kappa analysis. The difference between percentage diameter of stenosis and percentage area of stenosis was assessed with the paired Student’s t-test.
Results
The study population was composed of 147 patients who underwent PCI for 155 lesions between them. Table 1 shows the characteristics of the patients and the 155 lesions. Mean age of the patients was 64.7 years (range 28–95). There were 107 men (72.8%) and 42 women (27.2%).
Table 1. Characteristics of the patients and lesions.
| Characteristics | Total: 147 patients/155 lesions |
| Mean age, years | 64.7 ± 11.3 |
| Female, n (%) | 40 (27.2) |
| Male, n (%) | 107 (72.8) |
| Vessel | |
| LAD, n (%) | 68 (46.4) |
| Cx, n (%) | 39 (25.2) |
| RCA, n (%) | 42 (27.1) |
| Intermediate, n (%) | 2 (1.3) |
| Percentage stenosis | |
| Mean (range) | 84 (55–99) |
| Intervention, n | |
| Stent | 159 |
| Balloon | 5 |
| Stent type, n | |
| BMS | 92 |
| DES | 56 |
| BMS + DES | 2 |
| Stent size (mm) | |
| Length (mean) | 19.1 ± 6.6 |
| Diameter (mean) | 3.13 ± 0.49 |
| QCA | |
| Minimal lumen diameter (mm) | |
| Mean | 1.19 ± 0.48 |
| Range | 0.09–2.53 |
| Reference diameter (mm) | |
| Mean | 2.90 ± 0.58 |
| Range | 1.75–5.22 |
LAD, left anterior descending artery; Cx, circumflex artery; RCA, right coronary artery; BMS, bare-metal stent; DES, drug-eluting stent; QCA, quantitative coronary analysis.
The mean percentage of stenosis of the 155 lesions determined visually by the primary operator was 84% (range 55–99). The most commonly reported category for percentage of stenosis by the primary operator was 70–90%. The most treated vessel was the left anterior descending artery (LAD) (68, 46.4%), followed by the right coronary artery (RCA) (42, 27.1%), the circumflex artery (Cx) (39, 25.2%) and the intermediate artery (two, 1.3%).
In total, 159 stents were implanted. Five patients underwent balloon dilatation only, 92 underwent bare-metal stent implantation, whereas 56 had drug-eluting stent implantation. Both bare-metal and drug-eluting stents were implanted in two patients. Mean stent length was 19.1 ± 6.6 mm (range 8–54). Mean stent diameter was 3.13 ± 0.49 mm (range 2.0–4.75).
Mean percentages of stenosis determined by the primary, second and third operator by visual estimation were 84.0, 80.4 and 80.4%, respectively (Table 2). Concordance between the operators was evaluated with kappa (κ) analysis. There was a moderate degree of concordance in the categories 70–89% (κ: 0.406) and 90–99% (κ: 0.5813), while in the categories < 50 and 50–69%, there was a low degree of concordance between the operators (κ: 0.323 and κ: 0.261, respectively) (Table 3). Fig. 1.
Table 2. Visual estimations of three operators.
| Operators | Visual estimation, n (%) |
| Primary operator | |
| Percentage stenosis (mean) | 84.0 |
| > 50% | 0 (0) |
| 50–69% | 68 (3.9) |
| 70–89% | 75 (48.4) |
| 90–99% | 74 (47.7) |
| 2nd operator | |
| Percentage stenosis (mean) | 80.4 |
| < 50% | 3 (1.9) |
| 50–69% | 12 (7.7) |
| 70–89% | 82 (52.9) |
| 90–99% | 58 (37.4) |
| 3rd operator | |
| Percentage stenosis (mean) | 80.4 |
| < 50% | 3 (1.9) |
| 50–69% | 20 (12.9) |
| 70–89% | 73 (47.1) |
| 90–99% | 59 (38.1) |
Table 3. Evaluation of concordance between operators with kappa analysis.
| Group | Kappa | Concordance |
| < 50% | 0.261 | low–moderate |
| 50–69% | 0.406 | Moderate |
| 70–89% | 0.581 | Moderate |
| 90–99% | 0.323 | low–moderate |
| Total | 0.458 | moderate |
QCA was performed on all PCI-treated lesions by another operator who was blinded to the results of the visual assessment. The mean minimal lumen diameter was 1.19 ± 0.48 mm (range 0.09–2.53). The mean reference diameter was calculated as 2.90 ± 0.58 mm (range 1.75–5.22) and the mean length of the lesions was 17.3 ± 8.1 mm (range 6.7–45.1). Mean percentage diameter of stenosis was 58.4 ± 14.5% (range 29–97). Mean percentage area of stenosis was 80.6 ± 11.2% (range 50–99). The most commonly calculated category, mean percentage area of stenosis was 70–90%. There was a statistically significant difference between the QCA parameters percentage diameter of stenosis and percentage area of stenosis (58.4 ± 14.5% vs 80.6 ± 11.2%, p < 0.001).
The difference between the primary operator’s visual assessment and the QCA measurement was evaluated with the Student’s t-test. There was a statistically significant difference between the visual estimation of percentage of coronary stenosis, and the percentage diameter of stenosis and percentage area of stenosis determined by QCA (p < 0.01). Visual estimation of percentage of stenosis was higher than percentage diameter of stenosis and percentage area of stenosis calculated by QCA. A statistically significant difference was found between the stent size and reference diameter measured by QCA, and there was also a significant difference between stent length and lesion length determined by QCA (p < 0.001) (Table 4).
Concordance between visual estimation and QCA was investigated with kappa analysis. There was a low to moderate grade of concordance between the categories of visual estimation and the percentage area of stenosis (κ: 0.30) (Table 5) but there was no concordance between the categories of visual estimation and percentage diameter of stenosis on QCA (κ: –0.061) (Table 6). Of the 155 lesions considered above 70% on visual estimation, 23 were found by QCA not to be significant.
Table 4. Comparison between visual estimation and quantitative analysis.
| Visual analysis | Std | |||
| QCA estimation | Mean | Deviation | t-value | p-value |
| Percentage visual | 84.01 | 10.846 | 3.996 | 0.000** |
| Percentage minimum lumen area | 80.61 | 11.229 | ||
| Percentage visual | 84.01 | 10.846 | 25.440 | 0.000** |
| Percentage minimum lumen diameter | 58.42 | 14.513 | ||
| Stent diameter (visual) | 3.13 | 0.491 | 6.611 | 0.000** |
| Reference diameter | 2.91 | 0.586 | ||
| Stent length (visual) | 19.15 | 6.647 | 3.891 | 0.000** |
| Lesion length | 17.36 | 8.135 | ||
| Percentge area of stenosis (visual) | 80.61 | 11.229 | 60.500 | 0.000** |
| Percentage diameter of stenosis | 58.42 | 14.513 | ||
**p < 0.01.
Table 5. Comparison of concordance between visual estimation and percentage area of stenosis with kappa analysis.
| Percentage area of stenosis by QCA, n (%) | ||||||
| Visual percentage of stenosis | 50–69% | 70–89% | 90–99% | Total | Kappa | p-value |
| 50–69% | 2 (33.3) | 4 (66.7) | 0 (0) | 6 (100) | ||
| 70–89% | 17 (22.7) | 53 (70.7) | 5 (6.7) | 75 (100) | 0.300 | 0.000** |
| 90–99% | 6 (8.1) | 32 (43.2) | 36 (48.6) | 74 (100) | ||
| Total | 25 (15.6) | 89 (57.8) | 41 (26.6) | 155 (100) | ||
**p < 0.01
Table 6. Comparison of concordance between visual estimation and percentage diameter of stenosis with kappa analysis.
| Percentage diameter of stenosis by QCA, n (%) | ||||||
| Visual percentage of stenosis | n (%) | |||||
| < 50% | 50–69% | 70–89% | 90–99% | Kappa | p-value | |
| < 50% | 0 (0) | 0 (0) | 0 (0) | 0 (0) | ||
| 50–69% | 3 (50) | 0 (0) | 0 (0) | 0 (0) | ||
| 70–89% | 29 (53) | 42 (56) | 4 (38.7) | 0 (0) | –0.061 | 0.000** |
| 90–99% | 11 (14.9) | 31 (41.9) | 27 (36.5) | 5 (6.8) | ||
| Total | 43 (27.7) | 76 (49.0) | 31 (20.0) | 5 (3.2) | ||
**p < 0.01
Discussion
Many catheterisation laboratories still depend on visual estimation of lesion severity rather than quantitative analysis when deciding on PCI. Unfortunately, visual estimation may not be accurate and may vary between operators. Moreover, it has many limitations. The error with visual estimation may exceed 35%.10 Operators tend to overestimate severe stenosis, whereas modest stenosis is underestimated.11
In our study, we found a moderate degree of concordance between visual operators in the categories 70–89 and 90–99%. There was a low degree of concordance between visual operators in the categories < 50 and 50–69%. These results show that especially in cases of moderate and low degree of stenosis, interobserver variability increases.
QCA of coronary stenosis eliminates inter-observer bias and enables reproducible measurements. QCA is also useful for prediction of coronary restenosis after different coronary interventional techniques.12 It may also be used to follow the natural course of atherosclerosis. A decrease in the minimal lumen diameter and an increase in the percentage diameter of stenosis determined by QCA in follow-up coronary angiography was associated with increased coronary events. Change in minimal lumen diameter was the strongest predictor of coronary events.13
When we compared the results of visual estimation with QCA, we found significant differences between visual estimation and QCA in percentage diameter of stenosis and percentage area of stenosis. We also found differences between implanted stent diameter and reference diameter calculated by QCA and between stent length and lesion length derived from QCA. That means there is variability between implanted stent diameter and length and true size of the lesion. Physicians tended to implant larger and longer stents. The difference between mean diameter of implanted stent and mean reference diameter was 0.22 mm and the difference in mean length of the implanted stent and the lesion was 1.79 mm. Although statistically significant, this difference was not so great as to cause clinically important consequences. The important point is to cover the whole atherosclerotic segment with an optimal sized stent. Theoretically, choosing a longer stent size may increase the risk of stent restenosis in the future.
Twenty-three lesions considered significant according to visual estimation were found not to be significant when determined by QCA. This means that approximately 15% of patients, or one in seven, underwent unnecessary intervention.
When comparing the difference between percentage diameter of stenosis and percentage area of stenosis in determining the severity of stenosis, there was a statistically significant difference between the QCA-derived parameters (58.4 ± 14.5 vs 80.6 ± 11.2%). Percentage area of stenosis had a low to moderate grade of concordance with visual estimation, whereas there was no concordance between percentage diameter of stenosis and visual estimation. Percentage diameter of stenosis may underestimate the lesion.
In a study by Gottsauner-Wolf et al., it was shown that percentage area of stenosis more closely reflected the visual estimation of lesion severity than percentage diameter of stenosis.14 In another study, the authors used dobutamine stress echocardiography to determine the cut-off values of QCA parameters in estimation of the functional significance of coronary lesions. Angiographic cut-off values were determined as ≤ 1.07 mm, ≥ 75% and ≥ 52% for minimal lumen diameter, percentage area of stenosis and percentage diameter of stenosis, respectively. The cut-off value for percentage diameter of stenosis was much less than the cut-off value for percentage area of stenosis.15 Similar to the results of our study, percentage area of stenosis was prone to underestimate the lesion if the cut-off value was accepted as 70%. If percentage diameter of stenosis is used as QCA parameter, it may be more suitable to accept the cut-off value as 50%.
There are a few early trials comparing visual assessment with QCA. Older QCA software systems did not have the technology that we have today.2-8 Modern QCA software systems have advanced digital technology enabling more accurate and complex assessment.
There is only one recent study comparing visual assessment of severity of coronary lesions and QCA measurement. In this study, similar to our study, Nallamothu et al. showed that visual assessment tended to overestimate the lesion more than QCA. Inconsistency between QCA and visual assessment was high, especially in cases of moderately severe coronary lesions.9
QCA is a non-invasive and cheap method of quantification of coronary stenosis and measurement of reference vessel diameter for deciding the size of the stent. Despite its limitations, such as vessel foreshortening, it enables well-correlated measurements of lesion length, minimal lumen diameter and reference diameter. It also may prevent unnecessary PCI.
Conclusion
Visual estimation may overestimate a coronary lesion and may lead to unnecessary coronary intervention. There was low concordance in the categories < 50% and 50–69% between the operators. Percentage area of stenosis had a low to moderate grade of concordance with visual estimation. Percentage area of stenosis more closely reflected the visual estimation of lesion severity than percentage diameter of stenosis.
Contributor Information
Taner Sen, Email: medicineman_tr@hotmail.com.
Lale Dinc Asarcikli, Department of Cardiology, Dıskapi Yildirim Beyazit Education and Research Hospital, Ankara, Turkey.
Tolga Aksu, Department of Cardiology, Derince Education and Research Hospital, Derince, Turkey.
Habibe Kafes, Department of Cardiology, Yuksek Ihtisas Hospital, Ankara, Turkey.
Gokhan Gozubuyuk, Department of Cardiology, Malatya State Hospital, Malatya, Turkey.
References
- 1.Gensini GG, Kelly AE, Da Costa BCB. Quantitative angiography: the measurement of coronary vasomobility in the intact animal and man. Chest. 1971;60:522–530. doi: 10.1378/chest.60.6.522. [DOI] [PubMed] [Google Scholar]
- 2.Galbraith JE, Murphy ML, de Soyza N. Coronary angiogram interpretation. Interobserver variability. J Am Med Assoc. 1978;240(19):2053–2066. [PubMed] [Google Scholar]
- 3.Fisher LD, Judkins MP, Lesperance J, Cameron A, Swaye P, Ryan T. et al. Reproducibility of coronary arteriographic reading in the coronary artery surgery study (CASS) Cathet Cardiovasc Diagn. 1982;8:565–575. doi: 10.1002/ccd.1810080605. [DOI] [PubMed] [Google Scholar]
- 4.Goldberg RK, Kleiman NS, Minor ST, Abukhalil J, Raizner AE. Comparison of quantitative coronary angiography to visual estimates of lesion severity pre and post PTCA. Am Heart J. 1990;119:178–184. doi: 10.1016/s0002-8703(05)80098-5. [DOI] [PubMed] [Google Scholar]
- 5.Fleming RM, Kirkeeide RL, Smalling RW, Gould KL. Patterns in visual interpretation of coronary arteriograms as defected by quantitative coronary arteriography. J Am Coll Cardiol. 1991;18:945–951. doi: 10.1016/0735-1097(91)90752-u. [DOI] [PubMed] [Google Scholar]
- 6.Desmet W, Willems J, Lierde JV, Piessens J. Discrepancy between visual estimation and computer-assisted measurement of lesion severity before and after coronary angioplasty. Cathet Cardiovasc Diagn. 1994;31:192–198. doi: 10.1002/ccd.1810310306. [DOI] [PubMed] [Google Scholar]
- 7.Folland ED, Vogel RA, Hartigan P, Bates ER, Beauman GJ, Fortin T. et al. Relation between coronary artery stenosis assessed by visual, caliper, and computer methods and exercise capacity in patients with single-vessel coronary artery disease. The Veterans Affairs ACME Investigators. Circulation. 1994;89:2005–2014. doi: 10.1161/01.cir.89.5.2005. [DOI] [PubMed] [Google Scholar]
- 8.Leape LL, Park RE, Bashore TM, Harrison JK, Davidson CJ, Brook RH. Effect of variability in the interpretation of coronary angiograms on the appropriateness of use of coronary revascularization procedures. Am Heart J. 2000;139:106–113. doi: 10.1016/s0002-8703(00)90316-8. [DOI] [PubMed] [Google Scholar]
- 9.Nallamothu BK, Spertus JA, Lansky AJ, Cohen DJ, Jones PG, Kureshi F. et al. Comparison of clinical interpretation with visual assessment and quantitative coronary angiography in patients undergoing percutaneous coronary intervention in contemporary practice: the Assessing Angiography (A2) project. Circulation. 2013;127(17):1793–1800. doi: 10.1161/CIRCULATIONAHA.113.001952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.De Rouen TA, Murray JA, Owen W. Variability in the analysis of coronary arteriograms. Circulation. 1977;55 doi: 10.1161/01.cir.55.2.324. 324. [DOI] [PubMed] [Google Scholar]
- 11.Fleming RM, Kirkeeide RL, Smalling RW, Gould KL. Patterns in visual interpretation of coronary arteriograms as detected by quantitative coronary arteriography. J Am Coll Cardiol. 1991;18 doi: 10.1016/0735-1097(91)90752-u. 945. [DOI] [PubMed] [Google Scholar]
- 12.Serruys PW, Foley DP, Kirkeeide RL, King SB 3rd. Restenosis revisited: insights provided by quantitative coronary angiography. Am Heart J. 1993;126 doi: 10.1016/0002-8703(93)90689-7. 1243. [DOI] [PubMed] [Google Scholar]
- 13.Mack WJ, Xiang M, Selzer RH, Hodis HN. Serial quantitative coronary angiography and coronary events. Am Heart J. 2000;139(6):993–999. doi: 10.1067/mhj.2000.105702. [DOI] [PubMed] [Google Scholar]
- 14.Gottsauner-Wolf M, Sochor H, Moertl D, Gwechenberger M, Stockenhuber F, Probst P. Assessing coronary stenosis. Quantitative coronary angiography versus visual estimation from cine-film or pharmacological stress perfusion images. Eur Heart J. 1996;17(8):1167–1174. doi: 10.1093/oxfordjournals.eurheartj.a015033. [DOI] [PubMed] [Google Scholar]
- 15.Baptista J, Arnese M, Roelandt JR, Fioretti P, Keane D, Escaned J. et al. Quantitative coronary angiography in the estimation of the functional significance of coronary stenosis: correlations with dobutamine–atropine stress test. J Am Coll Cardiol. 1994;23:1434–1439. doi: 10.1016/0735-1097(94)90388-3. [DOI] [PubMed] [Google Scholar]