Skip to main content
NCBI home page
European Heart Journal Cardiovascular Imaging logoLink to European Heart Journal Cardiovascular Imaging
. 2015 Sep 15;17(7):756–764. doi: 10.1093/ehjci/jev229

Multi-laboratory inter-institute reproducibility study of IVOCT and IVUS assessments using published consensus document definitions

Edouard Gerbaud 1, Giora Weisz 2,3, Atsushi Tanaka 1, Manabu Kashiwagi 1, Takehisa Shimizu 2,3, Lin Wang 2,3, Christiano Souza 2,3, Brett E Bouma 1, Melissa J Suter 1, Milen Shishkov 1, Giovanni J Ughi 1, Elkan F Halpern 4, Mireille Rosenberg 1, Sergio Waxman 5, Jeffrey W Moses 2,3, Gary S Mintz 2,3, Akiko Maehara 2,3,†, Guillermo J Tearney 1,6,7,*
PMCID: PMC4907381  PMID: 26377904

Abstract

Aims

The aim of this study was to investigate the reproducibility of intravascular optical coherence tomography (IVOCT) assessments, including a comparison to intravascular ultrasound (IVUS). Intra-observer and inter-observer variabilities of IVOCT have been previously described, whereas inter-institute reliability in multiple laboratories has never been systematically studied.

Methods and results

In 2 independent laboratories with intravascular imaging expertise, 100 randomized matched data sets of IVOCT and IVUS images were analysed by 4 independent observers according to published consensus document definitions. Intra-observer, inter-observer, and inter-institute variabilities of IVOCT qualitative and quantitative measurements vs. IVUS measurements were assessed. Minor inter- and intra-observer variability of both imaging techniques was observed for detailed qualitative and geometric analysis, except for inter-observer mixed plaque identification on IVUS (κ = 0.70) and for inter-observer fibrous cap thickness measurement reproducibility on IVOCT (ICC = 0.48). The magnitude of inter-institute measurement differences for IVOCT was statistically significantly less than that for IVUS concerning lumen cross-sectional area (CSA), maximum and minimum lumen diameters, stent CSA, and maximum and minimum stent diameters (P < 0.001, P < 0.001, P < 0.001, P = 0.02, P < 0.001, and P = 0.01, respectively). Minor inter-institute measurement variabilities using both techniques were also found for plaque identification.

Conclusion

In the measurement of lumen CSA, maximum and minimum lumen diameters, stent CSA, and maximum and minimum stent diameters by analysts from two different laboratories, reproducibility of IVOCT was more consistent than that of IVUS.

Keywords: Reproducibility, Intravascular ultrasound imaging, Optical frequency domain imaging, Optical coherence tomography

Introduction

Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are widely available for evaluating high-resolution images of the coronary artery in vivo. Consensus document guidelines have been published for both techniques in order to harmonize their use and analysis.1–4 These two techniques are increasingly used for the assessment of the natural history of atherosclerosis, vascular remodelling, and pharmacological and percutaneous interventions.5–6 In a core laboratory setting, the inter- and intra-observer reproducibility for qualitative and quantitative measurements with both techniques has been previously established in many studies.7–11 Although one study underlines the necessity to centrally analyse IVUS data obtained in multi-centre studies,12 to the best of our knowledge, the evaluation of inter-institute reliability for IVOCT is currently lacking. Indeed, awareness of inter-institute differences may be particularly important in multi-centre pharmacological or percutaneous intervention trials. Accordingly, the purpose of this study was to investigate further inter- and intra-observer reproducibility, the inter-institute variability for IVOCT quantitative and qualitative measurements vs. IVUS measurements using published consensus document definitions.

Methods

Study population

Forty-two non-consecutive patients scheduled for elective percutaneous coronary intervention (PCI) were enrolled in two centres (Columbia University Medical Center, New York, NY, USA and Lahey Clinic Medical Center, Burlington, MA, USA). MGH, Columbia and Lahey Institutional Review Boards approved the study protocol. Patients with acute coronary syndrome, haemodynamic instability, renal insufficiency (glomerular filtration rate <50 mL/min), allergy to X-ray contrast, unprotected left main coronary artery disease, venous bypass graft lesions, chronic total occlusions, last remaining vessel, or extremely tortuous vessels were excluded. Patients underwent the following procedures in the catheterization laboratory: coronary angiography, PCI of the culprit lesion, and intravascular imaging in random order: IVOCT imaging and IVUS imaging. About 74 coronary arteries (left anterior descending artery, n = 28; left circumflex artery, n = 23; right coronary artery, n = 23) imaged from these 42 patients were studied. Ninety-six pullbacks on both native (primarily, n = 27) and stented coronary artery segments (pre-PCI, n = 26; post-PCI, n = 43) were included. To assess the inter-observer, intra-observer, and inter-institute variabilities of IVOCT quantitative and qualitative measurements vs. IVUS measurements, randomized matched data sets of 100 IVOCT and 100 IVUS intracoronary images were analyzed by 4 independent observers from 2 different laboratories (E.G. and M.K. for the Tearney laboratory, Massachusetts General Hospital, Boston, MA, USA; T.S. and L.W. for the Columbia University Medical Center, New York, NY, USA) who were blinded to other data (Table 1). These four observers had worked for at least 1 year as intravascular imaging researchers and were certified by their respective laboratories by completion of a common training programme.

Table 1.

Design of the study

Institute First read data set Second read data set (1 month later)
Observer 1 CRF IVUS H IVUS G
Observer 2 CRF OFDI O No
Observer 3 MGH IVUS A
OFDI J		 No
OFDI L
Observer 4 MGH IVUS F
OFDI P		 No
No
Open in a new tab

Intra-observer, inter-observer, and inter-institute variabilities were assessed as follows: intra-observer variability for IVUS measurements: IVUS H analysis vs. IVUS G analysis; intra-observer variability for OFDI measurements: OFDI J analysis vs. OFDI L analysis; inter-observer variability for IVUS measurements: IVUS H analysis vs. IVUS A analysis vs. IVUS F analysis; inter-observer variability for OFDI measurements: OFDI O analysis vs. OFDI J analysis vs. OFDI P analysis; inter-institute variability for IVUS measurements: IVUS H analysis vs. IVUS F analysis; inter-institute variability for OFDI measurements: OFDI O analysis vs. OFDI P analysis. For each set, the new order of images was obtained using web-based randomization software. MGH, Massachusetts General Hospital; CRF, Columbia Research Foundation.

IVUS acquisition

IVUS imaging was performed after intracoronary administration of nitrates (0.1–0.2 mg) using commercially available mechanical (iLab™ with 40 MHz Atlantis SR Pro catheters, Boston Scientific, Fremont, CA, USA) or phased-array transducer systems (s5™ with 20 MHz Eagle Eye Gold catheters, Volcano Therapeutics, Rancho Cordova, CA, USA), as described elsewhere in conventional manner, using an automated pullback device operating at 0.5 mm/s.13

IVOCT acquisitions

IVOCT imaging was performed with non-commercial frequency domain optical coherence tomography (FD-OCT) systems (Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, MA, USA), also known as optical frequency domain imaging (OFDI) systems, that operate and perform identically to commercial IVOCT systems, as described previously.14,15 This system used a wavelength-swept laser (center wavelength of ∼1310 nm) as a light source. The FD-OCT imaging catheter had a short monorail design with a catheter profile of 2.4 Fr, compatible with 6 F guiding catheters. The OCT acquisition technique was in line with recent expert review documents.3 During the flushing process, motorized pullback FD-OCT imaging was performed at a rate of 20 mm/s.

Matched IVUS–IVOCT image sets

The matched IVUS–IVOCT image sets were made by two independent interventional cardiologists (A.M. or A.T.) who have 10 years of experience in intravascular imaging. They generated Tiff stack files from original IVUS and OCT data. Complex geometries such as side-branch take-off/bifurcation carina were included. Images were preliminarily evaluated for diagnostic quality. Typical IVUS artefacts (i.e. non-uniform rotational distortion, air bubble, and geometric distortion due to the off-centred position of the IVUS probe in the artery) were excluded. Typical IVOCT artefacts (i.e. movement artefacts, flush defect artefacts, and fibre decentration and non-parallelism artefacts) were also excluded. IVUS and OCT images that were deemed to be of diagnostic quality were then co-registered by pullback distance and confirmed by using anatomical landmarks and following these successive steps: (i) the absolute landmarks were the left anterior descending coronary artery/left circumflex coronary artery bifurcation for the left coronary artery system or the atrioventricular node coronary artery/posterior descending coronary artery bifurcation for the right coronary artery system; (ii) additional anatomical landmarks, i.e. side branch or perivascular structure such as vein, and muscle were systematically used; (iii) morphological features in the cross-sectional image, including calcification shape, prominent vasa vasorum, and lumen morphology, were also used to confirm registration precision; (iv) stent features including post-stent or old stent cases permitted us to obtain corresponding images of IVUS and IVOCT; and (v) known pullback speed was integrated when pullbacks were considered quite stable (i.e. A.M. or A.T. did not recognize any non-linearities in the pullback rate) and confirmed the same length (<10% difference) between IVUS and IVOCT. In this database, sets of 100 IVUS–IVOCT matched images were generated using web-based randomization software.

IVUS data analysis

IVUS measurements, both geometric and compositional analyses, were made on a standalone computer workstation using ImageJ software.16 All quantitative and qualitative data were evaluated following published consensus document definitions.1,2 The lumen and vessel borders were traced manually for each image (Figure 1). To evaluate the intra-observer variability, one observer of the Columbia University Medical Center repeated the analysis of another set 1 month later. The following quantitative data were measured: lumen cross-sectional area (CSA), minimum and maximum luminal diameters, stent CSA, minimum and maximum stent diameters, external elastic membrane (EEM) area when identified and/or present under the lesion [excluding cross-sectional images that contain artefacts that obscure a significant portion (>90°)], atheroma CSA (defined by the EEM CSA minus lumen CSA), plaque burden [calculated as (atheroma CSA/EEM CSA) × 100 (%)], minimum and maximum atheroma thicknesses, atheroma eccentricity index [calculated as (maximum atheroma thickness – minimum atheroma thickness)/maximum atheroma thickness], the total arc of attenuation, and the total arc of calcium. The total arc of attenuation equals the sum of different arcs of attenuation in the same cross-section. The total arc of calcium equals the sum of different arcs of calcium in the same cross-section. According to consensus document definitions,1 plaque composition was also characterized in one of the following categories: hypoechoic, hyperechoic/isoechoic, calcified, or mixed. Echo-attenuated plaque was also identified by the absence of the ultrasound signal behind plaque that was either hypoechoic or isoechoic, but contained no bright calcium. Plaque rupture, thrombus, plaque protrusion, and incomplete stent apposition and dissection were also assessed.

Figure 1.

Figure 1

Open in a new tab

Example of IVUS measurements. Lumen and EEM areas are delineated. The minimum and maximum lumen diameters are illustrated using a double-headed arrow (open and solid arrowheads, respectively). Furthermore, the minimum and maximum atheroma thicknesses are illustrated using double-headed arrows (white for minimum and black for maximum).

IVOCT data analysis

Anonymized data were analysed on a standalone computer workstation using ImageJ software16 (Figure 2). All quantitative and qualitative data were evaluated following the published consensus document definitions.3,4 To evaluate the intra-observer variability, one observer of the Tearney laboratory repeated the analysis of another set 1 month later. The following quantitative data were measured: lumen CSA, minimum and maximum luminal diameters, stent CSA, minimum and maximum stent diameters, EEM area when identified and/or present under the lesion, atheroma CSA (defined by the EEM CSA minus lumen CSA), plaque burden [calculated as (atheroma CSA/EEM CSA) × 100 (%)], minimum and maximum atheroma thicknesses, atheroma or plaque eccentricity index, and internal elastic membrane (IEM) when identified and/or present under the lesion. The same lumen measurements as for EEM have been made for the IEM. All these measurements were obtained, if EEM and/or IEM was clearly identified in the IVOCT image (excluding EEM and/or IEM delineation when the image contains a significant obscure part on more than 90° of its circumference). According to consensus document definitions, plaque composition was also characterized in one of the following categories: fibroatheroma, fibrous plaque, and fibrocalcific plaque. The cut-off minimal cap thickness used to define the thin-capped fibroatheroma was 65 μm. The maximum lipid arc, the fibrous cap thickness (mean of three successive measurements), and the maximum calcium arcs were also measured. Plaque rupture, thrombus, prolapse, stent malapposition, dissection, macrophages within plaque, cholesterol crystals, and intimal vessels were also identified.3

Figure 2.

Figure 2

Open in a new tab

Example of IVOCT measurements. This IVOCT image was matched with the IVUS image presented in Figure 1. Lumen and EEM areas are traced. The minimum and maximum lumen diameters are illustrated using a double-headed arrow (open and solid arrowheads, respectively). Furthermore, the minimum and maximum atheroma thicknesses are illustrated using double-headed arrows (white for minimum and black for maximum).

Statistical analysis

Continuous data are expressed as mean ± standard deviation or median (interquartile range) when appropriate. Intra-observer and inter-institute variabilities for IVUS and IVOCT quantitative data were determined as mean (relative) difference (bias) and standard deviations, according to the methods of Bland and Altman. For each image, the magnitude of the difference observed between the two laboratories was computed as the absolute value for lumen CSA, minimum and maximum luminal diameters, stent CSA, minimum and maximum stent diameters, EEM area when identified and/or present under the lesion, atheroma CSA, plaque burden, minimum and maximum atheroma thicknesses, and atheroma or plaque eccentricity index. For comparisons within the magnitudes of inter-institute measurement differences for IVUS and IVOCT, a Wilcoxon signed rank test was performed. Inter-observer agreement for quantitative data of both techniques was assessed by intraclass correlation coefficient (ICC) based on the random-effects analysis of variance model. An ICC value greater than 0.90 was considered excellent. Analysis was performed using Cohen's kappa or Fleiss' kappa (where the number of observers is more than two) for categorical variables. A kappa value of 0.81–1.0 indicates almost perfect agreement, a value of 0.61–0.80 indicates substantial agreement, and a value of 0.41–0.60 indicates moderate agreement.17 P-values less than 0.05 were considered significant. All statistics were calculated using NCSS (NCSS 2001; NCSS Statistical software, Kaysville, UT, USA).

Results

Intra-observer variability for IVUS measurements

Intra-observer variability was very low for lumen CSA and minimum and maximum luminal diameters (−0.05 ± 0.25 mm2 and −0.03 ± 0.09 and −0.04 ± 0.21 mm, respectively) (Figure 3 and see Supplementary data online, Table S1). Likewise, the mean (standard deviation) differences were negligible for stent CSA and minimum and maximum stent diameters (0.04 ± 0.23 mm2 and 0.004 ± 0.13 and 0.03 ± 0.11 mm, respectively). Intra-observer variability was also low for EEM area, atheroma CSA, plaque burden, minimum atheroma thickness, maximum atheroma thickness, and atheroma eccentricity index (−0.15 ± 0.87 mm2, −0.10 ± 0.88 mm2, −0.02 ± 3.1%, 0.01 ± 0.06 mm, −0.02 ± 0.20 mm, and −0.02 ± 0.07, respectively). Bland and Altman showed a good agreement for the arcs measurement [mean difference (standard deviation): 1.2 ± 7.4° for the total arc of attenuation and mean difference (standard deviation): −3.8 ± 9.2° for the total arc of calcium]. Regarding qualitative data, the kappa values for intra-observer agreement on hypoechoic, hyperechoic, mixed, calcified, and echo-attenuated plaque characterization were 0.85, 0.92, 0.82, 0.90, and 0.78, respectively.

Figure 3.

Figure 3

Open in a new tab

Intra-observer variability of lumen CSA, stent CSA, and atheroma CSA for IVUS and IVOCT measurements. Middle line: mean difference; top and bottom dotted lines: mean +1.96 SD and mean −1.96 SD, respectively.

Intra-observer variability for IVOCT measurements

Intra-observer variability was very low for lumen CSA and minimum and maximum luminal diameters (0.04 ± 0.19 mm2, 0.03 ± 0.11 mm, and 0.04 ± 0.13 mm, respectively) (Figure 3 and see Supplementary data online, Table S2). Likewise, the mean differences (standard deviation) were negligible for stent CSA and minimum and maximum stent diameters (0.05 ± 0.26 mm2 and 0.02 ± 0.09 and 0.03 ± 0.13 mm, respectively). Intra-observer variability was also low for EEM area, atheroma CSA, plaque burden, minimum atheroma thickness, maximum atheroma thickness, and atheroma eccentricity index (0.13 ± 0.39 mm2, 0.08 ± 0.35 mm2, 0.4 ± 2.0%, 0.01 ± 0.04 mm, −0.02 ± 0.08 mm, and −0.02 ± 0.07, respectively). In the same way, mean differences (standard deviation) were low for IEM area, IEM atheroma CSA, IEM minimum atheroma thickness, IEM maximum atheroma thickness, and IEM atheroma eccentricity index (−0.06 ± 0.31 mm2, −0.10 ± 0.29 mm2, −0.01 ± 0.03 mm, −0.02 ± 0.04 mm, and −0.06 ± 0.16, respectively). Bland and Altman showed a good agreement for the lipid and calcium arc measurements (1.8 ± 6.8° and −2.4 ± 9.2°, respectively). Furthermore, the mean difference (standard deviation) for the fibrous cap thickness was −3.6 ± 11.6 μm. Regarding qualitative data, the kappa values for intra-observer agreement on fibroatheroma, fibrous, and fibrocalcific plaque characterization were 0.83, 0.84, and 0.86, respectively.

Inter-observer variability for IVUS measurements

The ICC was 0.98 [95% confidence interval (CI): 0.97–0.99] for lumen CSA, 0.94 (95% CI: 0.92–0.96) for minimum lumen diameter, and 0.95 (95% CI: 0.90–0.98) for maximum lumen diameter (see Supplementary data online, Table S3). The ICC was 0.98 (95% CI: 0.97–0.99) for stent CSA, 0.97 (95% CI: 0.94–0.99) for minimum stent diameter, and 0.95 (95% CI: 0.93–0.97) for maximum stent diameter. In the same way, inter-observer reproducibility was high for EEM area (ICC = 0.92; 95% CI: 0.89–0.95) and atheroma CSA (ICC = 0.90; 95% CI: 0.86–0.93) and good for plaque burden (ICC = 0.88; 95% CI: 0.83–0.92), minimum atheroma thickness (ICC = 0.73; 95% CI: 0.63–0.81), maximum atheroma thickness (ICC = 0.83; 95% CI: 0.79–0.89), and atheroma eccentricity index (ICC = 0.76; 95% CI: 0.67–0.83). The ICC for the total arc of attenuation was 0.78 (95% CI: 0.40–0.96), whereas the ICC for the total arc of calcium was 0.89 (95% CI: 0.77–0.95). Regarding qualitative data, the agreement was excellent for stent identification (κ = 1.0), EEM identification under the lesion (κ = 0.87), echo-attenuated plaque characterization (κ = 0.88), thrombus (κ = 1.0), prolapse (κ = 1.0), stent malapposition (κ = 1.0), and dissection (κ = 0.95) detection. Furthermore, the agreement was substantial for hypoechoic plaque (κ = 0.75), hyperechoic plaque (κ = 0.78), mixed plaque (κ = 0.70), and calcified plaque (κ = 0.80). No plaque rupture was identified.

Inter-observer variability for IVOCT measurements

The ICC was 0.99 (95% CI: 0.99–1.0) for lumen CSA, 0.96 (95% CI: 0.91–0.98) for minimum lumen diameter, and 0.99 (95% CI: 0.99–0.99) for maximum lumen diameter (see Supplementary data online, Table S4). The ICC was 0.99 (95% CI: 0.99–1.0) for stent CSA, 0.98 (95% CI: 0.97–0.99) for minimum stent diameter, and 0.99 (95% CI: 0.99–0.99) for maximum stent diameter. In the same way, inter-observer reproducibility was high for EEM area (ICC = 0.98; 95% CI: 0.96–0.99), atheroma CSA (ICC = 0.91; 95% CI: 0.87–0.93), plaque burden (ICC = 0.95; 95% CI: 0.91–0.98), and maximum atheroma thickness (ICC = 0.93; 95% CI: 0.90–0.95) and substantial for minimum atheroma thickness (ICC = 0.82; 95% CI: 0.78–0.88) and atheroma eccentricity index (ICC = 0.72; 95% CI: 0.61–0.80). Likewise, inter-observer reproducibility was high for IEM area (ICC = 0.99; 95% CI: 0.98–1.0), IEM atheroma CSA (ICC = 0.96; 95% CI: 0.91–0.99), and IEM maximum atheroma thickness (ICC = 0.98; 95% CI: 0.96–0.99) and good for minimum atheroma thickness (ICC = 0.82; 95% CI: 0.64–0.92) and atheroma eccentricity index (ICC = 0.87; 95% CI: 0.68–0.95). The ICC for the lipid arc was 0.81 (95% CI: 0.60–0.93), and the ICC for the total arc of calcium was 0.89 (95% CI: 0.79–0.96). However, the ICC for the fibrous cap thickness measurement was low (ICC = 0.48; 95% CI: 0.08–0.81). Regarding qualitative data, the agreement was excellent for stent identification (κ = 1.0), IEM identification under the lesion (κ = 0.81), thrombus (κ = 0.88), prolapse (κ = 1.0), stent malapposition (κ = 1.0), dissection (κ = 0.91), and the presence of cholesterol crystals (κ = 0.93). Furthermore, the agreement was substantial for EEM identification under the lesion (κ = 0.79), fibroatheroma (κ = 0.76), fibrous plaque (κ = 0.78), fibrocalcific plaque (κ = 0.80), macrophages presence (κ = 0.79), and intimal vessels detection (κ = 0.80). As for IVUS, no plaque rupture was detected.

Inter-institute variability for IVUS and IVOCT measurements

Inter-institute mean differences and standard deviations for quantitative IVUS and IVOCT geometrical measurements are shown in Table 2 and Figure 4. The magnitudes of measurement differences between the two institutes are compared in Table 3. EEM CSA was measured by both institute observers in 90% of the IVUS images and in 40% of the IVOCT corresponding images. In the 40 IVOCT images, in which EEM CSA was defined by both institutes' observers, EEM CSA could also be determined in all corresponding IVUS images. The magnitude of inter-institute measurement differences for IVOCT was statistically significantly less than that of inter-institute measurement differences for IVUS in the following assessments: lumen CSA, maximum and minimum lumen diameters, stent CSA, and maximum and minimum stent diameters. When IVOCT measurements were available (i.e. EEM was identified), a similar trend was observed for EEM CSA, atheroma CSA, plaque burden, and minimum and maximum atheroma thicknesses (Table 3). Bland and Altman showed a moderate agreement for the IVUS total arc of attenuation [mean difference (standard deviation): 3.8 ± 19.2°] and for the IVUS total arc of calcium [mean difference (standard deviation): −11.4 ± 15.1°]. Regarding IVUS qualitative data, the agreement was excellent for stent identification (κ = 1.0), prolapse (κ = 1.0), and stent malapposition (κ = 1.0). Furthermore, the agreement was substantial for thrombus (κ = 0.66) and dissection (κ = 0.65). In addition, the kappa values for inter-institute agreement on hypoechoic, hyperechoic, mixed, calcified, and echo-attenuated plaque characterization were 0.74, 0.78, 0.72, 0.87, and 0.78, respectively. Bland and Altman showed a moderate agreement for the IVOCT lipid and calcium arcs measurements (−5.3 ± 31.8° and 5.1 ± 24.4°, respectively). Furthermore, the mean difference (standard deviation) for the fibrous cap thickness was −7.9 ± 38.6 μm. Regarding qualitative data, the agreement was excellent for stent identification (κ = 1.0), prolapse (κ = 1.0), stent malapposition (κ = 1.0), macrophages presence (κ = 0.88), and the presence of cholesterol crystals (κ = 0.85). Furthermore, the agreement was substantial for thrombus (κ = 0.72), IEM identification under the lesion (κ = 0.73), dissection (κ = 0.78), and intimal vessels detection (κ = 0.71). In addition, the kappa values for inter-institute agreement on fibroatheroma, fibrous, and fibrocalcific plaque characterization were 0.76, 0.74, and 0.82, respectively.

Table 2.

Inter-institute reproducibility for quantitative IVUS and OFDI geometrical measurements

IVUS CRF Observer 1 IVUS MGH Observer 4 Mean difference IVUS OFDI CRF Observer 2 OFDI MGH Observer 4 Mean difference OFDI
Lumen CSA (mm2) 7.06 ± 2.81 6.87 ± 2.75 0.19 ± 0.65 6.86 ± 2.79 6.91 ± 2.90 −0.06 ± 0.32
Lumen max. diameter (mm) 3.10 ± 0.68 3.19 ± 0.63 −0.09 ± 0.22 3.13 ± 0.63 3.09 ± 0.66 0.04 ± 0.10
Lumen min. diameter (mm) 2.65 ± 0.52 2.63 ± 0.50 0.02 ± 0.28 2.64 ± 0.56 2.66 ± 0.60 −0.02 ± 0.17
Stent CSA (mm2) 7.94 ± 3.29 7.84 ± 3.30 0.10 ± 0.60 7.60 ± 3.26 7.65 ± 3.22 −0.05 ± 0.25
Stent max. diameter (mm) 3.31 ± 0.72 3.26 ± 0.75 0.05 ± 0.18 3.18 ± 0.64 3.15 ± 0.64 0.03 ± 0.10
Stent min. diameter (mm) 2.94 ± 0.63 2.98 ± 0.63 −0.04 ± 0.12 2.91 ± 0.62 2.92 ± 0.65 0.01 ± 0.08
EEM CSA (mm2)a 12.26 ± 2.97 12.06 ± 2.57 0.20 ± 1.11 11.07 ± 2.61 10.96 ± 2.92 0.11 ± 0.70
Atheroma CSA (mm2)a 5.79 ± 2.36 5.92 ± 2.30 −0.13 ± 1.05 4.83 ± 1.41 4.71 ± 1.43 0.11 ± 0.74
Plaque burden (%) 46.6 ± 14.3 48.8 ± 15.4 −2.2 ± 6.8 43.9 ± 11.7 43.3 ± 10.7 0.6 ± 3.3
Max. atheroma thickness (mm)a 0.85 ± 0.25 0.79 ± 0.25 0.06 ± 0.18 0.71 ± 0.27 0.74 ± 0.29 −0.03 ± 0.12
Min. atheroma thickness (mm)a 0.33 ± 0.14 0.29 ± 0.14 0.04 ± 0.14 0.27 ± 0.09 0.24 ± 0.08 0.03 ± 0.05
Atheroma eccentricity indexa 0.59 ± 0.16 0.61 ± 0.18 −0.02 ± 0.15 0.59 ± 0.17 0.62 ± 0.12 −0.03 ± 0.13
Open in a new tab

Values are expressed as mean ± standard deviation. Mean (relative) differences (bias) and standard deviations were calculated according to the method of Bland and Altman. CSA, cross-sectional area; EEM, external elastic membrane; max., maximal; min., minimal; MGH, Massachusetts General Hospital; CRF, Columbia Research Foundation. Atheroma CSA was calculated as EEM CSA – lumen CSA. Plaque burden was calculated as (atheroma CSA/EEM CSA) × 100 (%). Atheroma eccentricity index was calculated as: (maximum atheroma thickness – minimum atheroma thickness)/maximum atheroma thickness.

aIn the 40 OFDI images, in which EEM CSA was measured by both institute observers, EEM CSA could also be determined in all corresponding IVUS images.

Figure 4.

Figure 4

Open in a new tab

Inter-institute variability of lumen CSA, stent CSA, and atheroma CSA for IVUS and IVOCT measurements. Middle line: mean difference; top and bottom dotted lines: mean +1.96 SD and mean −1.96 SD, respectively.

Table 3.

Comparison between magnitudes of inter-institute measurement differences for quantitative IVUS and OFDI geometrical measurements

Inter-institute measurement differences magnitude for IVUS Inter-institute measurement differences magnitude for OFDI P-value
Lumen CSA (mm2) 0.33 (0.12–0.67) 0.10 (0.05–0.22) <0.001
Lumen max. diameter (mm) 0.16 (0.06–0.30) 0.06 (0.03–0.10) <0.001
Lumen min. diameter (mm) 0.12 (0.06–0.27) 0.04 (0.02–0.08) <0.001
Stent CSA (mm2) 0.26 (0.20–0.50) 0.17 (0.10–0.30) 0.02
Stent max. diameter (mm) 0.16 (0.09–0.20) 0.05 (0.03–0.09) <0.001
Stent min. diameter (mm) 0.10 (0.03–0.14) 0.04 (0.01–0.08) 0.01
EEM CSA (mm2)a 0.86 (0.39–1.28) 0.18 (0.05–0.36) 0.007
Atheroma CSA (mm2)a 0.68 (0.53–1.05) 0.17 (0.06–0.34) 0.02
Plaque burden (%)a 5.6 (2.2–7.3) 1.9 (0.4–2.1) 0.002
Max. atheroma thickness (mm)a 0.14 (0.07–0.20) 0.06 (0.03–0.14) 0.03
Min. atheroma thickness (mm)a 0.07 (0.03–0.10) 0.03 (0.01–0.05) 0.01
Atheroma eccentricity indexa 0.07 (0.04–0.15) 0.06 (0.04–0.11) 0.42
Open in a new tab

Values are expressed as median (interquartile range). CSA, cross-sectional area; EEM, external elastic membrane; max., maximal; min., minimal. Atheroma CSA was calculated as EEM CSA – lumen CSA. Plaque burden was calculated as (atheroma CSA/EEM CSA) × 100 (%). Atheroma eccentricity index was calculated as (maximum atheroma thickness – minimum atheroma thickness)/maximum atheroma thickness. P-value indicates the use of a Wilcoxon signed rank to compare IVUS and OFDI inter-institute mean differences magnitudes.

aIn the 40 OFDI images, in which EEM CSA was measured by both institute observers, EEM CSA could also be determined in all corresponding IVUS images.

Discussion

In this study, the main finding is that the inter-institute variability of measurements for IVOCT is statistically significantly less than that of measurements for IVUS in the following assessments: lumen CSA, maximum and minimum lumen diameters, stent CSA, and maximum and minimum stent diameters. Furthermore, intra- and inter-observer variability results for both techniques in this study were nearly similar to other studies previously published.

In our study, the inter-institute variability of geometrical measurements for IVOCT is statistically significantly less than that for IVUS. The first explanation is that the spatial resolution of IVOCT is greater than that of IVUS. Thus, the axial resolution ranges from 10 to 20 μm, compared with 80–100 μm for IVUS. Furthermore, the lateral resolution in IVOCT catheters is typically 30–50 μm when compared with 150–250 μm for IVUS.6 Another possible reason for this difference is the superior ability of OCT to visualize the lumen–intima interface compared with IVUS, therefore allowing OCT to visualize the true lumen dimensions, which IVUS can sometimes overestimate.18 This finding is consistent with that of Magnus et al.,19 who recently observed that IVUS inter-observer variability for measurement of in-stent CSA was significantly higher than IVOCT inter-observer variability (IVUS in-stent CSA: 1.34 mm2 vs. IVOCT in-stent CSA: 0.85 mm2; P = 0.024). Moreover, although there was a similar trend in favour of IVOCT for atheroma CSA, plaque burden, and atheroma minimum and maximum thicknesses, we acknowledge that EEM under the lesion was clearly less identified by IVOCT compared with IVUS, which is a bias in the results comparison. The reason that the EEM is more frequently identified in IVUS is that IVUS signal penetration (∼5 mm) is deeper than IVOCT penetration (∼2 mm) and IVOCT light is significantly attenuated by lipid. Consequently, although IVUS measurements are probably less consistent than IVOCT measurements, IVUS still appears to be the best technique for evaluating plaque burden and vascular remodelling, regardless of the composition of the plaque. Concerning plaque composition, the inter-institute agreement for both techniques yielded a substantial concordance. As demonstrated in this study, assessments by analysts from two different centres with the same training programme may result in statistically significant minor differences for plaque composition. In trials in which both technologies are being increasingly used and in which minor changes in plaque composition are expected,5 awareness of the inter-institute difference may be important in the design of a multi-centre study.

Using published consensus document definitions, our study evaluated intra- and inter-observer variabilities of both techniques regarding a wide and nearly complete range of quantitative and qualitative data. Previous studies were generally more focused on a specific analysis. Inter-observer IVUS assessments of plaque composition were highly correlated, except possibly for mixed plaque (κ = 0.70). Likewise, Palmer et al.9 showed a high level of agreement except for heterogeneous/mixed plaque (κ = 0.78). We suppose that part of this variability may be due to the definition of a mixed plaque. Inter-observer IVOCT reproducibility yielded a very good concordance for all geometry measurements, except for the fibrous cap thickness (ICC = 0.48). Similarly, Kim et al.20 observed equivalent ICC values, emphasizing that it remains challenging to detect the inner border of the lipid pool within the plaque for cap measurement. Finally, in our study, the reliability for macrophages, cholesterol crystals, and intimal vessels identification was also acceptable. To the best of our knowledge, this finding has never been reported.

Limitations

Our study included a small number of images (n = 100) per set. However, the analysis was complete, including a wide range of quantitative and qualitative data using published consensus document definitions. To generate the sets, images with poor quality were excluded, which is a bias of selection. The matching process used in this study may have some imperfections. The adoption of dedicated software can overcome this limitation, such as carpet view analysis of IVOCT images that has been recently described to enable a matching comparison of the same stent portion during serial time points.21 This study was conducted in two highly experienced centres in intravascular imaging in the USA. The results of this study may not be applicable elsewhere. Analyses were performed offline on selected images of coronary segments from patients with stable angina. Thus, our findings cannot be extrapolated to inter-institute studies in an online setting or in patients with acute coronary syndromes. Our non-commercial IVOCT system is similar to the Terumo FD-OCT imaging system (Lunawave, Terumo, Tokyo, Japan). Finally, parameters such as vessel location, vessel size, plaque burden, calcification, de novo vs. stented lesions, flushing media, and ECG cycle that could affect differences between IVOCT and IVUS assessments were not incorporated in the analysis.

Conclusion

In the measurement of lumen CSA, maximum and minimum lumen diameters, stent CSA, and maximum and minimum stent diameters by analysts from two different laboratories, inter-institute reproducibility of IVOCT was found to be more consistent than that of IVUS. Inter-institute agreement is substantial using both technologies for plaque composition. These findings may have important implications for the design of future studies that pool intravascular imaging parameters evaluated and measured by multiple institutions.

Supplementary data

Supplementary data are available at European Heart Journal — Cardiovascular Imaging online.

Funding

The research was funded in part by NIH R01HL076398.

Acknowledgements

E.G. has received a grant from the French Federation of Cardiology.

Conflict of interest: G.J.T.: Receives catheter materials from Terumo. Receives Royalties from Terumo and MIT and receives sponsored research funding from Canon Inc. and Infraredx. A.M.: ACIST and Boston Scientific Consultant.

References

  • 1.Di Mario C, Görge G, Peters R, Kearney P, Pinto F, Hausmann D et al. Clinical application and image interpretation in intracoronary ultrasound. Study Group on Intracoronary Imaging of the Working Group of Coronary Circulation and of the Subgroup on Intravascular Ultrasound of the Working Group of Echocardiography of the European Society of Cardiology. Eur Heart J 1998;19:207–29. [DOI] [PubMed] [Google Scholar]
  • 2.Mintz GS, Nissen SE, Anderson WD, Bailey SR, Erbel R, Fitzgerald PJ et al. American College of Cardiology clinical expert consensus document on standards for acquisition, measurement and reporting on intravascular ultrasound studies (IVUS). A report of the American College of Cardiology task force on clinical expert consensus documents. J Am Coll Cardiol 2001;37:1478–92. [DOI] [PubMed] [Google Scholar]
  • 3.Tearney GJ, Regar E, Akasaka T, Adriaenssens T, Barlis P, Bezerra HG et al. International Working Group for Intravascular Optical Coherence Tomography (IWG-IVOCT). Consensus standards for acquisition, measurement, and reporting of intravascular optical coherence tomography studies. A report from the international working group for intravascular optical coherence tomography standardization and validation. J Am Coll Cardiol 2012;59:1058–72. [DOI] [PubMed] [Google Scholar]
  • 4.Prati F, Guagliumi G, Mintz GS, Costa M, Regar E, Akasaka T et al. for the Expert's OCT Review Document. Expert review document part 2: methodology, terminology and clinical applications of optical coherence tomography for the assessment of interventional procedures. Eur Heart J 2012;33:2513–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Hartmann M, Huisman J, Böse D, Jensen LO, Schoenhagen P, Mintz GS et al. Serial intravascular ultrasound assessment of changes in coronary atherosclerotic plaque dimensions and composition: an update. Eur J Echocardiogr 2011;12:313–21. [DOI] [PubMed] [Google Scholar]
  • 6.Bezerra HG, Costa MA, Guaglimi G, Rollins AM, Simon DI. Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications. J Am Coll Cardiol Cardiovasc Interv 2009;2:1035–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Haussmann D, Lundkvist AJS, Friedrich GJ, Mullen WL, Fitzgerald PJ, Yock PG. Intracoronary ultrasound imaging: intraobserver and interobserver variability of morphometric measurements. Am Heart J 1994;128:674–80. [DOI] [PubMed] [Google Scholar]
  • 8.Blessing E, Hausmann D, Sturm M, Wolpers HG, Amende I, Mügge A. Intravascular ultrasound and stent implantation: intraobserver and interobserver variability. Am Heart J 1999;137:368–71. [DOI] [PubMed] [Google Scholar]
  • 9.Palmer ND, Northridge D, Lessells A, McDicken WN, Fox KA. In vitro analysis of coronary atheromatous lesions by intravascular ultrasound: reproducibility and histological correlation of lesion morphology. Eur Heart J 1999;20:1701–6. [DOI] [PubMed] [Google Scholar]
  • 10.Gonzalo N, Garcia-Garcia HM, Serruys PW, Commissaris KH, Bezerra H, Gobbens P et al. Reproducibility of quantitative optical coherence tomography for stent analysis. Eurointervention 2009;5:224–32. [DOI] [PubMed] [Google Scholar]
  • 11.Fedele S, Biondi-Zoccai G, Kwiatkowski P, Di Vito L, Occhipinti M, Cremonesi A et al. Reproducibility of coronary optical coherence tomography for lumen and length measurements in humans (The CLI-VAR [Centro per la Lotta contro l'Infarto-VARiability] study. Am J Cardiol 2012;110:1106–12. [DOI] [PubMed] [Google Scholar]
  • 12.Huisman J, Egede R, Rdzanek A, Böse D, Erbel R, Kochman J et al. Between-centre reproducibility of volumetric intravascular ultrasound radiofrequency-based analyses in mild-to-moderate coronary atherosclerosis: an international multicentre study. EuroIntervention 2010;5:925–31. [PubMed] [Google Scholar]
  • 13.Regar E, Werner F, Siebert U, Rieber J, Thesen K, Mudra H et al. Reproducibility of neointima quantification with motorized intravascular ultrasound pullback in stented coronary arteries. Am Heart J 2000;139:632–7. [DOI] [PubMed] [Google Scholar]
  • 14.Vakoc B, Yun S, de Boer J, Tearney GJ, Bouma BE. Phase-resolved optical frequency domain imaging. Opt Express 2005;13:5483–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Jamil Z, Tearney G, Bruining G, Sihan K, van Soest G, Lightart J et al. Interstudy reproducibility of the second generation, Fourier domain optical coherence tomography in patients with coronary artery disease and comparison with intravascular ultrasound: a study applying automated contour détection. Int J Cardiovasc Imaging 2013;29:39–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nat Methods 2012;9:671–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [PubMed] [Google Scholar]
  • 18.Kubo T, Akasaka T, Shite J, Suzuki T, Uemura S, Kozuma K et al. OCT compared with IVUS in a coronary lesion assessment. The OPUS-CLASS study. J Am Coll Cardiol Img 2013;6:1095–104. [DOI] [PubMed] [Google Scholar]
  • 19.Magnus PC, Jayne JE, Garcia-Garcia HM, Swart M, van Es GA, Tijssen J et al. Optical coherence tomography vs. intravascular ultrasound in the evaluation of observer variability and reliability in the assessment of stent deployment: The OCTIVUS Study. Catheter Cardiovasc Interv 2015;86:229–35. [DOI] [PubMed] [Google Scholar]
  • 20.Kim SJ, Lee H, Kato K, Yonetsu T, Xing L, Zhang S et al. Reproducibility of in vivo measurements for fibrous cap thickness and lipid arc by OCT. J Am Coll Cardiol Img 2012;5:1072–4. [DOI] [PubMed] [Google Scholar]
  • 21.Gabriele A, Marco V, Gatto L, Paoletti G, Di Vito L, Castriota F et al. Reproducibility of the carpet view system: a novel technical solution for display and off line analysis of OCT images. Int J Cardiovasc Img 2014;30:1225–33. [DOI] [PubMed] [Google Scholar]

Articles from European Heart Journal Cardiovascular Imaging are provided here courtesy of Oxford University Press

RESOURCES