Abstract
Aim:
To evaluate inter-core laboratory variability of quantitative coronary angiography (QCA) parameters in comparison with intra-core laboratory variability in a randomized controlled trial evaluating drug-eluting stents.
Methods:
A total of 50 patients with 62 coronary lesions were analyzed by four analysis experts belonging to an Angiographic Core Laboratory (ACL: 1 expert) and a Cardiovascular Imaging Core Laboratory (CICL: 3 experts). QCA was based on the same standard operating procedure, but selections of projection and cine frames were at the discretion of each analyst. Inter- and intra-core laboratory variabilities were evaluated by accuracy, precision, Bland Altman analysis, and coefficient of variation.
Results:
Pre-MLD (minimal lumen diameter) was significantly smaller in results from ACL than those from all CICL experts. Number of analyzed projections did not affect pre-MLD results. Acute gain was larger in ACL than in CICL2. No significant difference was observed in late loss and loss index between inter-core laboratories. Agreement between core labs in the Bland-Altman analysis for each QCA parameter was as follows (mean difference, 95% limits of agreement): pre-MLD (–0.32, –0.74 to 0.10), stent MLD (0.08, –0.28 to 0.44), acute gain (0.22, –0.44 to 0.88), and late loss (–0.07, –0.69 to 0.55). Agreement between analysts in CICL (mean difference, 95% limits of agreement) was: pre MLD (–0.03, –0.37 to 0.31), stent MLD (0.15, –0.15 to 0.45), acute gain (0.05, –0.45 to 0.55), and late loss (0.04, –0.52 to 0.60). The widest limits of agreement among three analyses were shown in both analyses. Width of limited agreement in the intra-core laboratory analysis tended to be smaller than the inter-core laboratory analysis with these parameters. Coefficient of variation tended to be larger in lesion length (LL), acute gain, late loss, and loss index in inter- and in intra- core laboratory comparisons.
Conclusion:
Inter-core laboratory QCA variability in late loss and loss index analysis could be similar to intra-core laboratory variability, but more strict alignment between core laboratories would be necessary for initial procedural data analysis.
Keywords: clinical trial, core laboratory, coronary angiography, drug-eluting stent, late loss, long-term, minimal lumen diameter, quantitative coronary angiography, validation
Introduction
Quantitative coronary angiography (QCA) has played a crucial role in evaluating interventional techniques and assessing the results of new technologies.1 Beyond angiographic metrics, recently developed angiography-derived fractional flow reserve (FFR) may be a useful tool for diagnosing ischemia-producing lesions in patients with non-complex coronary artery disease.1,2 Meanwhile, although QCA metrics such as acute gain and late lumen loss proved to be instrumental in assessing new technologies, it is true that there are still multiple factors that affect QCA results, including guide catheter size,3 contrast injection into the vessel, cine projection selection, number of cine projections, cine frame selection,4 and decision of reference vessel segment. During the QCA process, manual processes cannot be completely eliminated. Therefore, sources of variability could exist during the process of preparation of cine films and QCA; it would be still preferable to evaluate inter- and intra-observer reliability of QCA analysis in multicenter controlled trials despite the development of well-accepted and widely distributed QCA systems. Previous studies that evaluated inter- and intra- core laboratory (lab) variability of QCA results have been published mainly over the last 20 years.5–7 In the era of drug-eluting stent (DES), complex lesions such as diffuse, bifurcated, or chronic total lesions have been indicated for percutaneous coronary intervention (PCI) more frequently than balloon angioplasty or bare-metal stent owing to new devices and improved long-term results. For complex lesions, QCA analysis may be less accurate compared with simple analysis under the use of current QCA algorithms.8,9 Recently, an inter-core lab variability in analyzing QCA for bifurcation lesions was reported using dedicated software for bifurcations.10 Although these studies reported acceptable variability between core labs, the corresponding core labs used the same cine frame in exactly the same cine projections.7,10 That is, a comparison was performed under conditions favorable for QCA. However, manually selected cine series and frame selection might influence the results from the standpoint of practical analysis even in core labs in which experienced analysts analyze angiographic data using the same protocol. To date, no core lab validation study has been conducted for lesions, including complex ones, in which cine film series and cine frame were selected at the discretion of the analysis experts.
Methods
Subjects
The Japan-Drug Eluting Stents Evaluation: A Randomized Trial [J-DESsERT (J-D)] study is a prospective, multicenter, randomized, non-inferiority trial in terms of target vessel failure at 8 months after the index PCI procedure comparing Cypher™ sirolimus-eluting stent (SES) (Cordis/Johnson & Johnson, Miami Lakes, FL, USA) with Taxus™ paclitaxel-eluting stent (PES) (Boston Scientific Corp, Natick, MA, USA) in Japan.11 Patients eligible for implantation of DES were enrolled in J-D and randomly assigned to undergo PCI with either SES or PES. Of the 3533 patients enrolled in J-D between May 2008 and August 2010, follow-up angiography at 8 months after stent implantation was performed in 600 patients as a QCA sub-study. Among the QCA sub-group subjects, the initially registered consecutive 50 patients were used for this study.
Study background: validation of Japan Cardiovascular Imaging Core Lab as a core lab
The Angiographic Core Lab (ACL) at Brigham and Women’s Hospital was an independent core lab in the J-D study. When the J-D study started, CICL was organized by the sponsor of J-D (Association for Establishment of Evidence in Interventions) to promote randomized controlled trials in the field of interventional cardiology in Japan. CICL intended to validate QCA analysis quality by comparing QCA results with those from ACL using the same cine films registered in the J-D. This validation study was included in the J-D QCA sub-study. Three QCA analysis experts working in CICL participated in this study. The two core labs were aligned by the standard operating procedure (SOP) of QCA (Appendix).
The J-D study was approved by the institutional review board or medical ethics committee at each participating center, and written informed consent was obtained from all patients. The trial was registered on the http://www.clinicaltrials.gov website, with a unique identifier [ClinicalTrials.gov identifier: NCT00708669].
QCA methods
(1) The QCA protocol was based fundamentally on the SOP of ACL.7 QCA data analyzed at ACL was used as a control. CICL attempted to decrease the absolute differences as much as possible by strengthening the consensus on the SOPs. Three CICL experts (CICL1, 2, 3) subsequently analyzed the same cine films blindly to analyze data from ACL and other CICL experts.
(2) Cine angiographic acquisition protocol (guidelines to angiography) in the J-D study
At least a 6-Fr (diagnostic and/or guiding catheter) size was required as the reference source.3 Quality evaluation of cine angiograms taken at research sites and feedback to each site using feedback sheets were performed to improve and unify cine angiograms among research sites at the beginning of the J-D study.
(3) Cine projections and cine frames were selected independently by experts in ACL and CICL.
Inter- and intra- core lab variability in QCA analysis was evaluated using analysis data by four experts.
(4) QAngioXA version 6.0 (Medis, the Netherlands) was used in both core labs12 as an unified QCA system.
(5) Image calibration was performed using the contrast-filled diagnostic or guiding catheter (at least
6-Fr size).3
(6) Selection of cine projections and cine frames to analyze
During qualitative review of the cine angiograms the experts had selected, two (orthogonal if available) projections, demonstrating the normal reference segment and stenosis in an unforeshortened projections without overlap, were selected for analysis. However, in some cases, the number of views applicable for QCA was limited (the second view does not fulfill the criteria), and selection of projections was at the discretion of experts. The values of two projections were averaged. If one view was selected, the single number was adopted. Normal reference and minimal lesion diameters were averaged from two projections. Cine frame for each view was also selected by each expert in each core lab.
(7) Once image calibration had been performed, normal and diseased arterial segments were selected for analysis, and the computer-assisted edge detection algorithm was applied to obtain quantitative coronary dimensions.
(8) A 10-mm segment of normal reference vessel was calculated proximal and distal to the stenosis and was averaged for the determination of user-defined reference diameters.
(9) For the options of the QCA package, such as Gradient Field Transform (GFT),9 flagging was used at the discretion of experts.
(10) Manual collection
On the judgment of the operator, manual adjustments were permitted if necessary.
(11) QCA variables
QCA variables included minimal lumen diameter (MLD), mean reference diameter (mean R) (mean of proximal and distal reference diameters), % diameter stenosis (%DS) (calculated from MLD and mean of proximal and distal reference diameters), lesion length (LL), acute gain (post MLD–pre MLD), stent MLD, late loss (post stent MLD–follow-up stent MLD), and loss index (late loss/acute gain).
(12) When comparing QCA data between two core labs, ACL data was used as a control.
Statistical analysis
All continuous variables are expressed as mean ± standard deviation (SD). The ratio of selected projection numbers (single or two), and restenosis rate are expressed as categorical variables. The two studied comparison levels are expressed in terms of systematic and random errors, whereby the systematic error (accuracy) is defined by the average value of the signed differences between the individual analyses at ACL and CICL (inter-core lab) or measurements by CICL 1, 2, and 3 (intra-core lab). The random error (precision) is defined by the SD of the signed difference of the individual analyses at ACL and CICL1, 2, and 3 or measurements by CICL 1, 2, and 3. Coefficient of variation is defined as precision/mean value of the corresponding parameter to show the relative variation among different parameters. Inter- and intra-core lab analyses of agreement (relationship) between pairs of observations of the same lesion were analyzed using Bland–Altman plots and linear regression correlation. The following pairwise comparisons were made: (1) inter-core lab comparison (ACL versus CICL1, ACL versus CICL2, and ACL versus CICL3) and (2) intra-core lab comparison (CICL1 versus CICL2, CICL1 versus CICL3, and CICL2 versus CICL3).
For all continuous variables (MLD, mean R, LL, stent MLD, %DS, acute gain, late loss, loss index), normal distribution was evaluated using Shapiro–Wilk W test. For normally distributed variable, inter-core lab comparison of QCA variables was made using one-way analysis of variance (ANOVA) with Dunnett post hoc analysis among four experts analysis using data at ACL as control. Intra-core lab comparison was performed using one-way ANOVA with Tukey–Kramer HSD test among three experts. When skewed variables are detected in at least one group, multiple comparison was evaluated by nonparametric analysis using Kruskal–Wallis test with Steel test in inter-core lab analysis and with Steel–Dwass test in intra-observer analysis. For all continuous variables, comparative test for homogeneity of variance (equality of variances) between groups in precision was compared between groups using F-test. Proportional variables were compared using chi-squared test and test or Fisher’s exact test when applicable. Results are considered significant at p < 0.05. Statistical analysis was performed with JMP®10 (SAS Institute Inc., Cary, NC, USA).
Results
Baseline patient and lesion characteristics
This study consisted of 50 patients with 62 lesions (Table 1). The mean age of study subjects was 68.8 years; 15 patients had multiple vessel disease. Of the lesions, 36 (58.1%) were complex (types B2 and C), and 15 (24.2%) were type C lesions (lesion length: >20 mm) according to the American Heart Association/American College of Cardiology classification.13 Cypher stent or paclitaxel stents were implanted randomly according to the study protocol.11
Table 1.
Baseline patient and lesion characteristics.
| Patient number | 50 |
| Sex female | 13 |
| Age (years) | 68.8 ± 8.4 |
| Diagnosis (AP/UAP/SMI) | 40/4/6 |
| Number of diseased vessels (1/2/3) | 35/14/1 |
| Prior myocardial infarction | 9 |
| Prior CABG | 1 |
| Lesion characteristics | |
| Lesion number | 62 |
| Target vessel (LAD/LCx/LMT/RCA) | 28/17/0/17 |
| Lesion type (A/B1/B2/C) | 4/22/21/15 |
| Lesion length (mm) | 15.25 ± 6.05 |
| Ratio of lesion length >20 mm | 15 |
| Implanted drug-eluting stents | |
| Cypher (patients/lesions/stent number) | 24/30/31 |
| Taxus (patients/lesions/stent number) | 26/32/35 |
CABG, coronary artery bypass grafting.
Comparison of QCA variables between inter-core lab experts
Pre-MLD was significantly smaller in ACL expert than that for all CICL experts (Table 2). Pre-LL was smaller in ACL expert than that in CICL2 expert. ACL expert showed smaller post-MLD than CLCL3 expert and showed smaller post-mean R than CICL1 expert. Acute gain was larger in ACL expert than that in CICL2 expert, which was due mainly to the difference in pre-MLD. In the follow-up data, no significant difference was observed in all variables between inter-core lab experts. A significant difference was observed in pre-LL between CICL1 and CICL2, in post-mean R between CICL1 and CICL 3, and in post %DS between CICL1 and CICL3.
Table 2.
Baseline QCA data.
| Pre (n = 62) |
p value (versus ACL) |
Post (n = 62) |
p value (versus ACL) |
Follow up (n = 53) |
p value (versus ACL) |
||
|---|---|---|---|---|---|---|---|
| MLD (mm) | ACL | 0.75 ± 0.38 | NA | 2.51 ± 0.41 | NA | 2.31 ± 0.65 | NA |
| CICL1 | 1.04 ± 0.42 | 0.0001 | 2.62 ± 0.43 | 0.312 | 2.33 ± 0.64 | 0.998 | |
| CICL2 | 1.01 ± 0.38 | 0.0006 | 2.55 ± 0.45 | 0.914 | 2.28 ± 0.67 | 0.994 | |
| CICL3 | 1.07 ± 0.35 | <0.0001 | 2.69 ± 0.40 | 0.046 | 2.46 ± 0.65 | 0.518 | |
| mean R (mm) |
ACL | 2.63 ± 0.46 | NA | 2.62 ± 0.47 | NA | 2.75 ± 0.44 | NA |
| CICL1 | 2.72 ± 0.53 | 0.641 | 2.84 ± 0.51 | 0.038 | 2.87 ± 0.50 | 0.434 | |
| CICL2 | 2.65 ± 0.50 | 0.986 | 2.69 ± 0.49 | 0.757 | 2.76 ± 0.47 | 0.998 | |
| CICL3 | 2.71 ± 0.45 | 0.706 | 2.60 ± 0.47 | 0.993 | 2.82 ± 0.43 | 0.791 | |
| %DS (%) | ACL | 72.0 ± 11.9 | NA | 4.3 ± 9.8 | NA | 16.5 ± 18.6 | NA |
| CICL1 | 62.0 ± 13.0 | <0.0001 | 7.1 ± 7.3 | 0.216 | 18.9 ± 16.8 | 0.833 | |
| CICL2 | 62.1 ± 12.2 | <0.0001 | 4.8 ± 9.6 | 0.986 | 17.6 ± 20.1 | 0.979 | |
| CICL3 | 60.9 ± 10.3 | <0.0001 | 1.1 ± 8.9 | 0.124 | 13.2 ± 18.2 | 0.693 | |
| lesion length (mm) |
ACL | 15.25 ± 6.05 | NA | NA | NA | NA | NA |
| CICL1 | 15.22 ± 6.28 | 1.0 | NA | NA | NA | NA | |
| CICL2 | 18.53 ± 6.93 | 0.014 | NA | NA | NA | NA | |
| CICL3 | 16.05 ± 6.32 | 0.829 | NA | NA | NA | NA | |
| stent (mm) | ACL | NA | NA | 2.89 ± 0.40 | NA | 2.80 ± 0.55 | NA |
| CICL1 | NA | NA | 2.96 ± 0.45 | 0.673 | 2.82 ± 0.59 | 0.996 | |
| CICL2 | NA | NA | 2.82 ± 0.43 | 0.629 | 2.70 ± 0.55 | 0.702 | |
| CICL3 | NA | NA | 2.92 ± 0.42 | 0.952 | 2.81 ± 0.55 | 0.999 | |
| acute gain (mm) |
ACL | NA | NA | 1.76 ± 0.42 | NA | NA | NA |
| CICL1 | NA | NA | 1.59 ± 0.45 | 0.066 | NA | NA | |
| CICL2 | NA | NA | 1.54 ± 0.45 | 0.014 | NA | NA | |
| CICL3 | NA | NA | 1.63 ± 0.39 | 0.215 | NA | NA | |
| late loss (mm) |
ACL | NA | NA | NA | NA | 0.24 ± 0.43 | NA |
| CICL1 | NA | NA | NA | NA | 0.34 ± 0.41 | 0.573 | |
| CICL2 | NA | NA | NA | NA | 0.31 ± 0.53 | 0.806 | |
| CICL3 | NA | NA | NA | NA | 0.30 ± 0.50 | 0.875 | |
| loss index | ACL | NA | NA | NA | NA | 0.14 ± 0.28 | NA |
| CICL1 | NA | NA | NA | NA | 0.23 ± 0.31 | 0.377 | |
| CICL2 | NA | NA | NA | NA | 0.19 ± 0.38 | 0.831 | |
| CICL3 | NA | NA | NA | NA | 0.18 ± 0.33 | 0.908 |
ACL, Brigham and Women’s Hospital angiographic core laboratory; CICL, cardiovascular imaging core laboratory; %DS, % diameter stenosis; MLD, minimal lumen diameter; NA, not available; QCA, quantitative coronary angiography.
Accuracy, precision, coefficient of variation, agreement in Bland–Altman analysis, and correlation coefficient in all variables between inter- and intra- core lab experts
In accuracy, differences in values were very small in almost all groups except pre-MLD (–0.26 ± 0.20 to −0.32 ± 0.21) and %DS (9.94 ± 7.67 to 11.1 ± 7.63) in inter-core lab analysis (Table 3). In precision, the value was a little smaller in intra-core lab than in inter-core lab analysis in pre-MLD (0.14–0.17 versus 0.20–0.21), pre-mean R (0.16–0.18 versus 0.20–0.24), %DS (6.03–6.63 versus 7.67–8.88), stent MLD (0.11–0.15 versus 0.15–0.18), and acute gain (0.23–0.25 versus 0.25–0.33). Values were similar in LL (3.89–6.03 versus 4.48–4.88), late loss (0.20–0.32 versus 0.23–0.31), and loss index (0.15–0.22 versus 0.17–0.23). The agreement between core labs in the Bland-Altman analysis for each QCA metric was as follows (mean difference, 95% limits of agreement): pre-MLD (–0.32, –0.74 to 0.10), stent MLD (0.08, –0.28 to 0.44), acute gain (0.22, –0.44 to 0.88), and late loss (–0.07, –0.69 to 0.55). The agreement between analysts in CICL was as follows (mean difference, 95% limits of agreement): pre MLD (–0.03, –0.37 to 0.31), stent MLD (0.15, –0.15 to 0.45), acute gain (0.05, –0.45 to 0.55), and late loss (0.04, –0.52 to 0.60). The widest limits of agreement among three analyses was shown in both inter-core lab and intra-core lab analyses. Overall, the value of coefficient of variation (CV) had similar tendencies between intra-core lab and inter-core lab variations (a little smaller in intra-core lab variation than inter-core lab variation) for all parameters. CV was as small as 0.10 in pre-mean R, pre-%DS, and stent MLD. CV of pre-MLD, LL, and acute gain was between 0.15 and 0.36. CV of late loss and loss index was largest (0.66–1.39). In correlation coefficient, each value was >0.80 except for pre-%DS (0.729–0.814) and LL (0.691–0.738) in inter-core lab and only LL (0.586–0.811) in intra-core lab. In pre-MLD, pre-mean R, stent MLD, late loss, and loss index values were high (R > 0.80) in both inter- and intra- core lab analyses. The correlation curves are shown in Figure 1 (A, B) (pre-MLD and stent MLD) and in Figure 2 (A, B) (acute gain and late loss) side by side. The Bland–Altman plots between the two groups are shown in Figure 3 (A, B) (pre-MLD and stent MLD) arranging inter-core lab and intra-core lab data side-by-side (pre-MLD, stent MLD, and follow-up MLD) and in Figure 4 (A, B) (acute gain and late loss). CV was smallest in pre-mean R and stent MLD and largest in late loss and loss index for both inter-core lab and intra-core lab analyses.
Table 3.
Accuracy, precision, CV, correlation coefficient.
| Inter-core lab | Intra-core lab | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Accuracy | Precision | CV |
R
p value |
Accuracy | Precision | CV |
R
p value |
|||
| pre MLD (mm) |
ACL –CICL1 |
–0.29 | 0.20 | 0.22 | 0.884 <0.0001 |
CICL1 –CICL2 |
0.03 | 0.16 | 0.16 | 0.927 <0.0001 |
| ACL –CICL2 |
–0.26 | 0.20 | 0.22 | 0.854 <0.0001 |
CICL1 –CICL3 |
–0.03 | 0.17 | 0.16 | 0.921 <0.0001 |
|
| ACL –CICL3 |
–0.32 | 0.21 | 0.25 | 0.836 <0.0001 |
CICL2 –CICL3 |
–0.06 | 0.14 | 0.13 | 0.925 <0.0001 |
|
| pre mean R (mm) |
ACL –CICL1 |
–0.09 | 0.21 | 0.08 | 0.918 <0.0001 |
CICL1 –CICL2 |
0.06 | 0.17 | 0.06 | 0.950 <0.0001 |
| ACL –CICL2 |
–0.02 | 0.24 | 0.09 | 0.876 <0.0001 |
CICL1 –CICL3 |
0.01 | 0.16 | 0.06 | 0.955 <0.0001 |
|
| ACL –CICL3 |
–0.08 | 0.20 | 0.07 | 0.903 <0.0001 |
CICL2 –CICL3 |
–0.05 | 0.18 | 0.07 | 0.937 <0.0001 |
|
| pre % DS (%) |
ACL –CICL1 |
9.94 | 7.67 | 0.11 | 0.814 <0.0001 |
CICL1 –CICL2 |
–0.06 | 6.28 | 0.10 | 0.877 <0.0001 |
| ACL –CICL2 |
9.89 | 8.88 | 0.13 | 0.729 <0.0001 |
CICL1 –CICL3 |
1.17 | 6.63 | 0.11 | 0.863 <0.0001 |
|
| ACL –CICL3 |
11.11 | 7.63 | 0.11 | 0.774 <0.0001 |
CICL2 –CICL3 |
1.22 | 6.19 | 0.10 | 0.862 <0.0001 |
|
| lesion length (mm) |
ACL –CICL1 |
0.01 | 4.86 | 0.32 | 0.691 <0.0001 |
CICL1 –CICL2 |
–3.32 | 6.03 | 0.36 | 0.586 <0.0001 |
| ACL –CICL2 |
–3.31 | 4.88 | 0.29 | 0.726 <0.0001 |
CICL1 –CICL3 |
–0.83 | 3.89 | 0.25 | 0.811 <0.0001 |
|
| ACL –CICL3 |
–0.80 | 4.48 | 0.29 | 0.738 <0.0001 |
CICL2 –CICL3 |
2.48 | 5.33 | 0.31 | 0.681 <0.0001 |
|
| stent MLD (mm) |
ACL –CICL1 |
–0.07 | 0.16 | 0.05 | 0.932 <0.0001 |
CICL1 –CICL2 |
0.15 | 0.15 | 0.05 | 0.941 <0.0001 |
| ACL –CICL2 |
0.08 | 0.18 | 0.06 | 0.913 <0.0001 |
CICL1 –CICL3 |
0.04 | 0.11 | 0.04 | 0.971 <0.0001 |
|
| ACL –CICL3 |
–0.03 | 0.15 | 0.05 | 0.933 <0.0001 |
CICL2 –CICL3 |
–0.11 | 0.14 | 0.05 | 0.947 <0.0001 |
|
| acute gain (mm) |
ACL CICL1 |
0.17 | 0.25 | 0.15 | 0.835 <0.0001 |
CICL1 –CICL2 |
0.05 | 0.25 | 0.16 | 0.843 <0.0001 |
| ACL –CICL2 |
0.22 | 0.33 | 0.20 | 0.717 <0.0001 |
CICL1 –CICL3 |
–0.04 | 0.24 | 0.15 | 0.852 <0.0001 |
|
| ACL –CICL3 |
0.13 | 0.28 | 0.17 | 0.760 <0.0001 |
CICL2 –CICL3 |
–0.09 | 0.23 | 0.15 | 0.864 <0.0001 |
|
| late loss (mm) |
ACL –CICL1 |
–0.10 | 0.23 | 0.79 | 0.847 <0.0001 |
CICL1 –CICL2 |
0.03 | 0.32 | 0.98 | 0.801 <0.0001 |
| ACL –CICL2 |
–0.07 | 0.31 | 1.13 | 0.814 <0.0001 |
CICL1 –CICL3 |
0.04 | 0.28 | 0.88 | 0.828 <0.0001 |
|
| ACL –CICL3 |
–0.06 | 0.29 | 1.07 | 0.815 <0.0001 |
CICL2 –CICL3 |
0.01 | 0.20 | 0.66 | 0.924 <0.0001 |
|
| loss index |
ACL –CICL1 |
–0.09 | 0.17 | 0.92 | 0.848 <0.0001 |
CICL1 –CICL2 |
0.04 | 0.22 | 1.05 | 0.812 <0.0001 |
| ACL –CICL2 |
–0.04 | 0.23 | 1.39 | 0.801 <0.0001 |
CICL1 –CICL3 |
0.05 | 0.16 | 0.78 | 0.874 <0.0001 |
|
| ACL –CICL3 |
–0.03 | 0.18 | 1.13 | 0.842 <0.0001 |
CICL2 –CICL3 |
0.01 | 0.15 | 0.81 | 0.918 <0.0001 |
|
ACL, Brigham and Women’s Hospital angiographic core laboratory; CICL, cardiovascular imaging core laboratory; CV, coefficient of variation ; %DS, % diameter stenosis; MLD, minimal lumen diameter.
Figure 1.
Correlation of pre-MLD (a, b) and stent MLD (c, d) between two analysis experts in inter- and intra-core lab analyses. Inter-core lab (a, c) and intra-core lab (b, d) correlations are shown side by side for visual comparison. In intra-core lab correlation curves, the horizontal axis shows CICL 1, 2 and vertical axis CICL 2,3, respectively. A good correlation is present between the two groups at follow up in both inter- and intra- core lab comparisons, with an R value of > 0.90. MLD pre- and post- stent shows a slightly lower R value of < 0.90 but > 0.80 in inter-core lab comparison.
a, ACL; c1, CICL1; c2, CICL2; c3, CICL3; MLD, minimal lumen diameter.
Figure 2.
Correlation of acute gain (a, b) and late loss (c, d) between analysis experts in inter- and intra-core lab analyses. Inter-core lab (a, c) and intra-core lab (b, d) correlations are shown side by side for visual comparison. In intra-core lab correlation curves, horizontal axis shows CICL 1, 2 and vertical axis CICL 2,3, respectively. A good correlation is present between the two groups at follow up in both inter- and intra- core lab comparison, with an R value of > 0.80, except for acute gain with a lower R value of 0.72.
a, ACL; c1, CICL1; c2, CICL2; c3, CICL3.
Figure 3.
Bland–Altman plots of pre-MLD (a, b) and stent MLD (c, d) by inter- and intra-core lab analyses. The differences between the two core labs (panels on the left) and between the two analysis experts in CICL (panels on the right) are plotted against the mean values for each data set. Three data sets are superimposed in this graph with different colored marks. The mean value is on the horizontal axis plotted against the indicated differences on the vertical axis. The solid lines represent the mean difference, whereas the dashed lines indicate the 95% limits of agreement (2×SD of the mean difference). Note that the comparison of the mean difference (accuracy) and precision between inter-core lab (c) and intra-core lab (d) analysis was similar in stent MLD, with less wide limits of agreement in intra-core lab (b) in pre-MLD.
a, ACL; c1, CICL1; c2, CICL2; c3, CICL3; MLD, minimal lumen diameter.
Figure 4.
Bland–Altman plots of acute gain (a, b) and late loss (c, d) in inter- and intra-core lab analyses. The differences between the two core labs (panels on the left) and between two analysis experts in CICL (panels on the right) are plotted against the mean values for each data set. Three data sets are superimposed in this graph with different colored marks. The mean value is on the horizontal axis plotted against the signed differences on the vertical axis. The solid lines represent the mean difference, whereas the dashed lines indicate the 95% limits of agreement (2×SD of the mean difference). Note that the comparison of the mean difference (accuracy) and precision in inter-core lab (c) and intra-core lab (d) analysis was similar in late loss with less wide limits of agreement in acute gain in the intra-core lab (b) analysis.
a, ACL; c1, CICL1; c2, CICL2; c3, CICL3; MLD, minimal lumen diameter.
Coincident rate of the projection numbers between inter- and intra-core lab experts
The coincident rate of the projection number tended to be higher in intra-core lab than in inter-core lab pre-and post-PCI (Table 4). Projection number increased post-PCI and at follow up in intra- and inter-core lab analysis. The ratio of projection number (single or two) was significantly higher in CICL1 than ACL and CICL2 in pre-PCI. It was significantly higher in CICL1 than in ACL in post-PCI and at follow up. Thus, it depended on each expert but not on each core lab. Significant difference in pre-MLD existed between ACL and CICL experts whether or not projection number was coincident in both laboratories (Table 5). This means projection number (single or two) did not affect the difference in pre-MLD between inter- and intra-core lab experts.
Table 4.
Coincident rate of the projection number between two core laboratories and between analysis experts.
| Inter core laboratories | |||
|---|---|---|---|
| Pre | Post | Follow-up | |
| ACL versus CICL1 | 49 (79.0%) | 53 (85.5%) | 43 (81.1%) |
| ACL versus CICL2 | 44 (71.0%) | 47 (75.8%) | 44 (83.0%) |
| ACL versus CICL3 | 45 (72.6%) | 53 (85.5%) | 43 (81.1%) |
| Intra core laboratory | |||
| Pre | Post | Follow-up | |
| CICL1 versus CICL2 | 51 (82.3%) | 52 (83.9%) | 44 (83.0%) |
| CICL1 versus CICL3 | 56 (90.3%) | 58 (93.5%) | 49 (92.5%) |
| CICL2 versus CICL3 | 53 (85.5%) | 56 (90.3%) | 46 (86.8%) |
| Percent of projection number | |||
| (Pre) | 2 | 1 | mean |
| ACL | 50 (80.6%) | 12 (19.4%) | 1.81 |
| CICL1 | 61 (98.4%) | 1 (1.6%) | 1.98 |
| CICL2 | 50 (80.6%) | 12 (19.4%) | 1.81 |
| CICL3 | 55 (88.7%) | 7 (11.3%) | 1.89 |
| (Post) | 2 | 1 | mean |
| ACL | 51 (82.3%) | 11 (17.7%) | 1.82 |
| CICL1 | 60 (96.8%) | 2 (3.2%) | 1.97 |
| CICL2 | 50 (80.6%) | 12 (19.4%) | 1.81 |
| CICL3 | 56 (90.3%) | 6 (9.7%) | 1.90 |
| (Follow up) | 2 | 1 | mean |
| ACL | 41 (77.4%) | 12 (22.6%) | 1.77 |
| CICL1 | 51 (96.2%) | 2 (3.8%) | 1.96 |
| CICL2 | 42 (79.2%) | 11 (20.8%) | 1.79 |
| CICL3 | 49 (92.5%) | 4 (7.5%) | 1.92 |
ACL, Brigham and Women’s Hospital angiographic core laboratory; CICL, cardiovascular imaging core laboratory.
Table 5.
Comparison of pre MLD between ACL and CICL.
| Overall | Coincident projection no. | Non coincident projection no. | |
|---|---|---|---|
| ACL | 0.75 ± 0.38 (n = 62) | 0.75 ± 0.38 (n = 49) | 0.76 ± 0.40 (n = 13) |
| CICL1 | 1.04 ± 0.42 (n = 62) | 1.03 ± 0.42 (n = 49) | 1.06 ± 0.41 (n = 13) |
| p value | <0.0001 | <0.0001 | 0.0003 |
| ACL | 0.75 ± 0.38 (n = 62) | 0.72 ± 0.40 (n = 44) | 0.83 ± 0.33 (n = 18) |
| CICL2 | 1.01 ± 0.38 (n = 62) | 0.98 ± 0.40 (n = 44) | 1.07 ± 0.35 (n = 18) |
| p value | <0.0001 | <0.0001 | 0.0005 |
| ACL | 0.75 ± 0.38 (n = 62) | 0.77 ± 0.37 (n = 45) | 0.70 ± 0.40 (n = 17) |
| CICL3 | 1.07 ± 0.35 (n = 62) | 1.09 ± 0.32 (n = 45) | 1.00 ± 0.43 (n = 17) |
| p value | <0.0001 | <0.0001 | 0.0001 |
ACL, Brigham and Women’s Hospital angiographic core laboratory; CICL, cardiovascular imaging core laboratory; MLD, minimal lumen diameter.
Comparison of restenosis rate between two core lab individuals
The restenosis rates at follow up were 4.8% (three cases) in ACL and CICL3, 8.1% (five cases) in CICL1, and 6.5% (four cases) in CICL2. No significant difference was observed among experts (p = 0.854). The same three cases showed >50% in % DS in all experts. Three cases revealed >50% only in CICL 1 (50.6%, 50.9%) and 2 (69.2%), respectively.
Discussion
Inter-core lab variability of late lumen loss was comparable with intra-core lab variability in 62 lesions treated with DES in the J-D study under the conditions of same SOP and QCA system but free projection and cine frame selection. No significant difference was observed in restenosis rate between two core labs. In contrast, pre-MLD showed a significant difference between two core labs. These results suggest that QCA results of follow-up data (late loss, loss index, and restenosis rate) from different core labs using the same SOP in the era of DES could be rendered as not significantly different even if performed under free cine projection and cine frame selection. However, QCA results of initial procedural data need more strict alignment in QCA analysis between core labs.
Strength of this QCA variability study
The strengths of this study are as follows. (1) The basic QCA protocol was in alignment with SOP in the two core labs but cine projections and frame number of each projection were at the discretion of each expert. Thus, this study could evaluate the influence of these on QCA. Difference in selection of the angiographic view itself is a major determinant of variability in QCA.14 Although it has been shown that differences in frame selection from the same angiographic view do not influence the accuracy and variability to a large extent,15 selection was limited to the end-diastolic phase. Our recent study showed the influence of cine frame selection on the QCA results under blind cine frame selection.4 The target lesions included complex type B2 and type C lesions (58.2%), which could be the potential source of variance between experts. These features might have yielded differences between our study and previous studies that also evaluated inter- core lab variability.5,6 (3) Follow-up data, including late loss, loss index, and restenosis rate, which are representative and important parameters in the interventional cardiology and obtained from pre, post, and follow-up parameters, were also evaluated. (4) The same QCA system and algorithm were used in both labs to eliminate the variability due to different systems.5,8,10,12
Variability of follow-up data between the two core labs
Late luminal loss has been proposed as a robust marker for evaluating DES in the overall population (either randomized or observational),16,17 which is more reliable than restenosis rates at discriminating the effectiveness of different DES. In this study, although there were small differences in %DS and restenosis rate between two core labs, we think that these differences did not significantly affect clinical prognosis, because the two core labs showed similar late loss and restenosis rates at 8 month follow up. Inter core-lab variability (systemic and random errors) was similar to intra- core lab variability and remained within a level equivalent to that of previous studies. Thus, this study suggests that the two independent core labs might give the same QCA results of follow-up data for DES treatments if the same SOP and QCA system were used at the DES treatment stage.
Variability in initial data between the two core labs
In contrast, a significant difference was observed between the two core labs in pre-MLD and LL. The difference in LL could be due to projection selection and cine frame selections.4,18 Because the difference in LL was detected also between intra-core lab analysis, these results might be due to each expert, rather than to the core lab. Cardiac construction may cause vessel flexion, which leads to change in LL. Determined reference sites might affect lesion length in very diffuse lesions with tapering. Because most non- foreshortening views, in which LL was measured, usually could be selected easily, the key projection should not be different among experts so frequently. Thus, selection of projection may not explain much LL variance. The use of three-dimensional QCA could be a potential alternative to lessen inter- and intra-core lab variability in LL, because accurate and robust reconstruction of the vessel centerline is achieved and the reproducibility of its applications, for example, the assessments of obstruction length and optimal viewing angle, is guaranteed.2,19
Difference in projection number and its effects on QCA outcome
We assumed that the number of projection(s) might have affected the MLD measurements because a single view could show larger %DS than mean of two orthogonal views in eccentric lesions. However, this was not the case in our study. Each technician selected two suitable projections (ideally orthogonal) and the frame for each projection blinded from each other. Perfect single projection, or another projection with some compromise, was at the discretion of experts in both core labs. There was a little difference in ratio of projection number (single or two) among four experts. It seemed to depend on the experts rather than on the core labs. The coincident rate of projection number did not affect QCA results in MLD in our study. The results of our study confirm a recent report that showed that the worst view versus the average views provided similar results regarding percent diameter stenosis.20
Limitations
This study has several limitations. First, consistency in measuring complex and/or tight lesions might have been inadequate despite the consistency of algorithm and basic protocol of QCA SOP. More rigorous alliance in performing QCA analysis such as GFT and flagging might have decreased the difference of pre-MLD values between inter-core labs.9 Although the dose and frequency of manual contour corrections, GFT, and flagging would be important for inter-core lab variability, they were not recorded in this study. Second, the sample size of the study was small, and we did not perform a power calculation to evaluate the difference in QCA parameters between inter- and intra-core labs. The justification of angiographic core laboratory as a reference might have been inadequate. We cannot completely exclude the possibility of significant difference in other parameters besides those that already showed significant difference by adopting a larger number of lesions.
Conclusion
There has been a continuing need for a central core lab in multicenter trials to optimize consistency. In some large studies, multiple central core labs could be required. Our study showed that inter-core lab QCA variability, including follow-up data from different core labs, could be similar to intra-core lab variability using same SOP and system, despite free projection and frame selection, but results of initial procedural data need more strict alignment between core labs.
Supplemental Material
Supplemental material, Appendix_Organized_descriptions_regarding_QCA_analysis for Inter- and intra-core laboratory variability in the quantitative coronary angiography analysis for drug-eluting stent treatment and follow up by Shigenori Ito, Kanako Kinoshita, Akiko Endo, Ryoko Kami, Yuko Kotake and Masato Nakamura in Therapeutic Advances in Cardiovascular Disease
Acknowledgments
We thank Jeffrey Popma and Alexandra Almonacid for their cooperation in the J-D QCA sub-study and providing the SOP for QCA methodology in the J-D study.
Footnotes
Conflict of interest statement: The authors declare that there is no conflict of interest.
Funding: The authors received no financial support for the research, authorship, and/or publication of this article.
ORCID iD: Shigenori Ito 
https://orcid.org/0000-0002-3319-1697
Supplemental material: Supplemental material for this article is available online.
Contributor Information
Shigenori Ito, Division of Cardiology, Sankuro Hospital, 7-80 Kosaka-cho, Toyota-shi, Aichi-ken, 471-0035, Japan; Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
Kanako Kinoshita, Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
Akiko Endo, Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
Ryoko Kami, Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
Yuko Kotake, Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
Masato Nakamura, Division of Cardiology, Toho University Medical Center Ohashi Hospital, Tokyo, Japan; Japan Cardiovascular Imaging Core Laboratory, Tokyo, Japan.
References
- 1. Collet C, Grundeken MJ, Asano T, et al. State of the art: coronary angiography. EuroIntervention 2017; 13: 634–643. [DOI] [PubMed] [Google Scholar]
- 2. Masdjedi K, van Zandvoort LJC, Balbi MM, et al. Validation of 3-dimensional quantitative coronary angiography based software to calculate fractional flow reserve: fast assessment of stenosis severity (FAST)-study. EuroIntervention. Epub ahead of print 14 May 2019. DOI: 10.4244/EIJ-D-19-00466. [DOI] [Google Scholar]
- 3. Ito S, Kinoshita K, Endo A, et al. Impact of catheter size on reliability of quantitative coronary angiographic measurements (comparison of 4Fr and 6Fr catheters). Heart Vessels 2016; 31: 1752–1757. [DOI] [PubMed] [Google Scholar]
- 4. Ito S, Kinoshita K, Endo A, et al. Impact of cine frame selection on quantitative coronary angiography results. Clin Med Insights Cardiol 2019; 13: 1179546819838232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Beauman GJ, Reiber JH, Koning G, et al. Comparisons of angiographic core laboratory analyses of phantom and clinical images: interlaboratory variability. Cathet Cardiovasc Diagn 1996; 37: 24–31. [DOI] [PubMed] [Google Scholar]
- 6. Sirnes PA, Myreng Y, Molstad P, et al. Reproducibility of quantitative coronary analysis, assessment of variability due to frame selection, different observers, and different cinefilmless laboratories. Int J Card Imaging 1996; 12: 197–203. [DOI] [PubMed] [Google Scholar]
- 7. Tuinenburg JC, Koning G, Hekking E, et al. One core laboratory at two international sites, is that feasible? An inter-core laboratory and intra-observer variability study. Catheter Cardiovasc Interv 2002; 56: 333–340. [DOI] [PubMed] [Google Scholar]
- 8. Desmet W, De Scheerder I, Beatt K, et al. In vivo comparison of different quantitative edge detection systems used for measuring coronary arterial diameters. Cathet Cardiovasc Diagn 1995; 34: 72–80; discussion 81. [DOI] [PubMed] [Google Scholar]
- 9. van der Zwet PM, Reiber JH, Koning G, et al. A new approach for the quantification of complex lesion morphology: the gradient field transform; basic principles and validation results accuracy and precision of quantitative digital coronary arteriography: observer-, short-, and medium-term variabilities angiographic assessment of dimensions of 6F and 7F mallinckrodt softouch coronary contrast catheters from digital and cine arteriograms. J Am Coll Cardiol 1994; 24: 216–224. [DOI] [PubMed] [Google Scholar]
- 10. Grundeken MJ, Ishibashi Y, Genereux P, et al. Inter-core lab variability in analyzing quantitative coronary angiography for bifurcation lesions: a post-hoc analysis of a randomized trial. JACC Cardiovasc Interv 2015; 8: 305–314. [DOI] [PubMed] [Google Scholar]
- 11. Nakamura M, Muramatsu T, Yokoi H, et al. Outcomes of the largest multi-center trial stratified by the presence of diabetes mellitus comparing sirolimus-eluting stents (SES) and paclitaxel-eluting stents (PES) in patients with coronary artery disease. The Japan drug-eluting stents evaluation: a randomized trial (J-DESsERT). Cardiovasc Interv Ther 2015; 30: 103–114. [DOI] [PubMed] [Google Scholar]
- 12. Hausleiter J, Nolte CW, Jost S, et al. Comparison of different quantitative coronary analysis systems: Artrek, CAAS, and CMS. Cathet Cardiovasc Diagn 1996; 37: 14–22; discussion 23. [DOI] [PubMed] [Google Scholar]
- 13. Theuerle J, Yudi MB, Farouque O, et al. Utility of the ACC/AHA lesion classification as a predictor of procedural, 30-day and 12-month outcomes in the contemporary percutaneous coronary intervention era. Catheter Cardiovasc Interv 2018; 92: E227–E234. [DOI] [PubMed] [Google Scholar]
- 14. Reiber JH, van Eldik-Helleman P, Kooijman CJ, et al. How critical is frame selection in quantitative coronary angiographic studies? Variabilities in measurement of coronary arterial dimensions resulting from variations in cineframe selection. Eur Heart J 1989; 10(Suppl. F): 54–59. [DOI] [PubMed] [Google Scholar]
- 15. Reiber JH, van Eldik-Helleman P, Visser-Akkerman N, et al. Variabilities in measurement of coronary arterial dimensions resulting from variations in cineframe selection. Cathet Cardiovasc Diagn 1988; 14: 221–228. [DOI] [PubMed] [Google Scholar]
- 16. Harrison RW, Radhakrishnan V, Lam PS, et al. Rationale and design of the east-west late lumen loss study: comparison of late lumen loss between Eastern and Western drug-eluting stent study cohorts. Am Heart J 2016; 182: 103–110. [DOI] [PubMed] [Google Scholar]
- 17. Mauri L, Orav EJ, Candia SC, et al. Robustness of late lumen loss in discriminating drug-eluting stents across variable observational and randomized trials. Circulation 2005; 112: 2833–2839. [DOI] [PubMed] [Google Scholar]
- 18. Jost S, Deckers J, Nikutta P, et al. Influence of the selection of angiographic projections on the results of coronary angiographic follow-up trials. International nifedipine trial on antiatherosclerotic therapy investigators. Am Heart J 1995; 130: 433–439. [DOI] [PubMed] [Google Scholar]
- 19. Tu S, Koning G, Jukema W, Reiber JH. Assessment of obstruction length and optimal viewing angle from biplane x-ray angiograms. Int J Cardiovasc Imaging 2010; 26: 5–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Hideo-Kajita A, Wopperer S, Beyene SS, et al. Impact of two formulas to calculate percentage diameter stenosis of coronary lesions: from stenosis models (phantom lesion model) to actual clinical lesions. Int J Cardiovasc Imaging 2019; 35: 2139–2146. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, Appendix_Organized_descriptions_regarding_QCA_analysis for Inter- and intra-core laboratory variability in the quantitative coronary angiography analysis for drug-eluting stent treatment and follow up by Shigenori Ito, Kanako Kinoshita, Akiko Endo, Ryoko Kami, Yuko Kotake and Masato Nakamura in Therapeutic Advances in Cardiovascular Disease