Validation of a semi-automatic software for optical coherence tomography – analysis in heart transplanted patients

Abstract

Optical Coherence Tomography (OCT) is an intravascular imaging modality enabling detailed evaluation of cardiac allograft vasculopathy (CAV) after heart transplantation (HTx). However, its clinical application remains hampered by time-consuming manual quantitative analysis. We aimed to validate a semi-automated quantitative OCT analysis software (Iowa Coronary Wall Analyzer, ICWA-OCT) to improve OCT-analysis in HTx patients. 23 patients underwent OCT evaluation of all three major coronary arteries at 3 months (3M) and 12 months (12M) after HTx. We analyzed OCT recordings using the semiautomatic software and compared results with measurements from a validated manual software. For semi-automated analysis, 31,228 frames from 114 vessels were available. The validation was based on a subset of 4287 matched frames. We applied mixed model statistics to accommodate the multilevel data structure with method as a fixed effect. Lumen (minimum, mean, maximum) and media (mean, maximum) metrics showed no significant differences. Mean and maximum intima area were underestimated by the semi-automated method (β-method_mean = − 0.289 mm², p < 0.01; β-method_max = − 0.695 mm², p < 0.01). Bland–Altman analyses showed increasing semi-automatic underestimation of intima measurements with increasing intimal extent. Comparing 3M to 12M progression between methods, mean intimal area showed minor underestimation (β-method_mean = − 1.03 mm², p = 0.04). Lumen and media metrics showed excellent agreement between the manual and semi-automated method. Intima metrics and progressions from 3M to 12M were slightly underestimated by the semi-automated OCT software with unknown clinical relevance. The semi-automated software has the future potential to provide robust and time-saving evaluation of CAV progression.

Introduction

Survival after Heart Transplantation (HTx) has improved over the last three decades with a median survival of 15–16 years [1]. This improvement is primarily driven by better outcomes within the first year after HTx. However, Cardiac Allograft Vasculopathy (CAV) remains a prevailing complication with a prevalence of 47% 10 years after HTx [2]. CAV gradually limits blood flow in the coronary vessels due to lumen narrowing as a consequence of diffuse global concentric intimal thickening. Ultimately, the reduced blood flow causes heart failure. CAV is considered the main cause of delayed mortality in HTx patients [3].

Routine clinical monitoring of CAV development predominantly relies on coronary lumen assessment by coronary angiography. Such evaluation often underestimates the disease progression since the early intima proliferation causes expansive vascular remodeling with limited luminal reduction [4, 5]. Initial CAV progression can be inhibited by therapy with mTOR inhibitors such as Everolimus [6]. Therefore, early diagnosis is crucial. Many transplant centers apply intravascular ultrasound (IVUS) to obtain more sensitive data regarding vessel layer thickness and composition in the coronary arteries [7, 8]. Optical Coherence Tomography (OCT) is a potentially superior imaging modality with high sensitivity that offers a tenfold increase in spatial resolution compared to IVUS [4, 9–11]. Recent studies have found convincing results in OCT detection of early CAV development with intimal thickening and luminal reduction [12–15]. However, manual vessel layer segmentation analysis is time-consuming and future prognostic clinical utility of OCT imaging in HTx relies on streamlining the offline quantitative process [16]. Machine learning can be used to generate automatic algorithms for detection of characteristic structures such as vessel layer borders, based on real life data.

In the presented study, we aimed to validate a novel machine-learning based semi-automated software, Iowa Coronary Wall Analyzer for OCT (ICWA-OCT), developed for robust and time-saving quantitative OCT analysis in heart transplanted patients.

Methods

Patients

All patient clinical data were originally collected for use in a previous study and are provided in Table 1, [12]. The study was designed as a retrospective validation study with the following inclusion criteria: age ≥ 18 years, heart transplanted at Aarhus University Hospital from July 2013 through January 2016, and written informed consent executed in accordance with the principles of the Helsinki Declaration. Exclusion criteria were creatinine > 200 μmol/liter and < 10 analyzable frames. The Central Denmark Region Committees on Biomedical Research Ethics approved the study. The study was registered with clinicaltrials.gov (NCT02077764).

Table 1.

Patient characteristics

	3M (N = 23 patients)	12M (N = 23 patients)	p-value
Men (%)	18 (78)		–
Age at HTx (years)	56 [40 to 64]		–
Reason for HTx
Dilated cardiomyopathy [n (%)]	11 (48)		–
Hypertrophic cardiomyopathy [n (%)]	1 (4)
Ischemic heart disease [n (%)]	5 (22)		–
Other [n (%)]	6 (25)		–
Cold ischemic time (minutes)	177 ± 43		–
Donor age (years)	51 [42 to 53]		–
Diabetes (%)	3 (13)	2 (9)	0.32
Hypertension (%)	16 (70)	16 (70)	1.00
Medication
Prednisolone (%)	23 (100)	23 (100)	1.00
Cyclosporine (%)	0 (0)	1 (4)	0.32
Tacrolimus (%)	23 (100)	22 (96)	0.32
Everolimus (%)	3 (13)	2 (9)	0.32
Mycophenolate mofetil (%)	23 (100)	23 (100)	1.00
Statins (%)	21 (91)	21 (91)	1.00
ACE/AT II inhibitor (%)	14 (61)	14 (61)	1.00
Furosemide / bumetanide (%)	5 (22)	5 (22)	1.00
Calcium channel blocker (%)	4 (17)	6 (26)	0.16
Aspirin (%)	3 (13)	3 (13)	1.00
Biochemistry
Creatinine (μmol/liter)	99 [81 to 117]	82 [74 to 110]	0.03
Hemoglobin (mmol/liter)	7.9 [7.3 to 8.4]	8.1 [7.5 to 8.5]	0.02
Total cholesterol (mmol/liter)	5.01 ± 1.1	5.09 ± 1.2	0.69
Troponin-T (ng/liter)	24 [17 to 35]	12 [6 to 16]	< 0.01
NT-pro-BNP (ng/liter)	669 [463 to 1031]	255 [161 to 474]	< 0.01

Data presented as number (%), median [interquartile range] or mean ± standard deviation. ACE angiotensin-converting enzyme; AT II angiotensin II; HTx heart transplantation; NT-pro-BNP N-terminal pro–brain

natriuretic peptide. 3M; 3 months, 12M; 12 months

Image acquisition

OCT scans were performed using Lunawave Optical Frequency Domain Imaging (Terumo Corp., Japan) at routine surveillance coronary angiography outpatient visit at 3 months (3M) and 12 months (12M) after HTx. Pullback speed was adjusted to optimize the scan time to 3 to 4 s while performing manual contrast injection. In the case of poor image quality, the recordings were repeated after adjustment of the guide catheter. Recordings were obtained for the left anterior descending artery (LAD) including the left main stem (LM), the left circumflex artery (LCX) and the right coronary artery (RCA). All OCT scans were exported in DICOM format for offline semi-automatic analysis, while RAW format was used for manual analysis.

Semi‑automatic analysis

The novel semi-automated software analyzes the whole OCT pullback and combines automatic image processing with subsequent minimally-interactive correction. The ICWA-OCT software employs machine learning as well as graph-based optimization components working in concert and was developed at the University of Iowa (Fig. 1) [13, 17]. It offers multi-layered image segmentation of the coronary wall (1), pullback registration of 3M to 12M pullbacks (2) and invalid angular section exclusion (3). See step by step approach below:

Fig. 1.

Iowa Coronary Wall Analyzer for OCT (ICWA-OCT). a Enlarged cross-sectional view of non-edited frame. b Enlarged cross-sectional view of vessel layer segmentation suggested by the semi-automatic software. Lumen is found within the red line, intima layer between red and purple lines and media layer between purple and green lines. c Cross-sectional view of angular analysis performed by the semi-automatic software. Included angles are displayed by the outmost red dotted circle (angular analysis)

1. For each OCT image frame, lumen, internal and external elastic membrane contours were automatically segmented using a fully 3D LOGISMOS graph-based approach [18, 19]. All automatically identified surfaces were then efficiently edited by experts using the computer-aided refinement method called Just-Enough-Interaction (JEI) (Fig. 1) [17, 20].

2. Registration of 3M and 12M pullback pairs with respect to location and orientation of corresponding OCT images was based on at least 3 matching landmark pairs such as side branches, microvessels and distinctive plaques. Pullback registration was obtained via linear interpolation of OCT images positioned between the matching landmarks. Linear extrapolation based on two closest landmarks was used proximally or distally outside of the landmark pairs.

3. Portions of the vessel wall in OCT images are frequently not analyzable due to e.g., guidewire shadow, excessive atherosclerosis reducing light penetration, blood artifacts, etc. A deep-learning method was trained using expert-defined valid/invalid OCT regions of coronary wall. OCT image data were processed by a fully convolutional neural network, in which strided layers, sigmoid activation and thresholding of the output determine angular regions of valid/invalid appearance for each OCT image. Furthermore, angular regions were assessed visually and—if needed—improved manually by correcting deep-learned angles until full satisfaction.

4. Step 3 was only applied for intimal thickness assessment.

Manual analysis

A customized version of QCU-CMS software (Medis Medical Imaging, Leiden, the Netherlands) based on a validated OCT segmentation approach [21], was applied for the manual analyses performed by Clemmensen et al. in their previous work [12]. Importantly, only segments with no side branches and no atherosclerosis were included for the manual layer segmentation to avoid imprecise comparison of undefinable vessel wall elements.

Manual quantitative cross-sectional OCT analysis was performed for every 0.5–0.6 mm corresponding to every 5^th frame generating a lumen-intima interface contour, a intima-media interface contour and a media-adventitia interface contour. Included vessel layer measurements were lumen area, intima area, intima thickness and media area.

Interpolation of the contours was performed at the guidewire shadow. Visual regions of the vessel wall had to constitute at least 2/3 of the circumference for a frame to be included. Layer borders were fitted in the not visual parts by extrapolation of adjacent visual parts. Pairing of manual analyses were based on the same frame registration as described for the semi-automated method to ensure comparable frames.

Endpoint definitions

All minimum, mean and maximum values were derived from the minimum, mean and maximum measurements for each pullback. Intima thickness was calculated for each frame as the distance between the lumen contour and the internal elastic membrane, relative to lumen center. Intima, media and lumen area were derived from multi-layer segmentation as differences of area between relevant layer borders.

Considering that the semi-automated analysis and the manual analysis produce different amounts of analyzed frames for each pullback, a matched frame analysis using only the frames included for manual analysis constitutes our primary validation outcome. Additionally, we performed a vessel-level comparison including all available frames from semi-automated analysis (S1–S2).

Statistical methods

For demographics, normally distributed continuous data are presented as mean ± standard deviation (SD) and compared using paired t-tests. Histograms and Q-Q plots were used on continuous data to check for normality. Logarithmic transformation was applied if histograms and Q-Q plots did not show normality in distribution. Non-normally distributed data are presented as median followed by interquartile range (IQR) and compared using Wilcoxon signed-rank tests. Categorical data are presented as count and percentage and compared using McNemar’s test. Medians and IQR are presented for all independent measurements by each method in supplementary tables S5–S7.

Two-sided p < 0.05 was considered to be statistically significant.

Scatterplots are used for visual demonstration of alignment of results derived from each method, whereas Bland–Altman (BA) plots are applied to depict systematic biases as mean differences.

The main statistical comparisons are based on multivariate mixed effect models accommodating the multilevel structure of the data with the aim to include several measurements per patient. The models include choice of method (semi-automatic or manual) as a fixed effect while patient and vessel levels are considered random effects. We adjust for time between 3M and 12M scans (ΔTime), except when analyzing disease progression (Table 3). All pullbacks are considered as individual observations and metrics are analyzed separately. The overall mixed model can be written:

$$\text{Measurement}=\text{Method}+\Delta \text{ Time}+\left[\text{Vessel}+\text{Patient}\right]$$

Maximum Likelihood (ML) is used as estimation method. Mixed model assumptions are tested. If random effects deviate from normal distribution by visual assessment (but still approximate normality), we apply the sandwich estimator [22]. The final models have been confirmed by Likelihood Ratio tests.

Table 3.

Numerical and mixed model comparison of delta values using matched frames

Matched frames (4287)	Semi-automatic	Manual	β-method	95%-CI	p
Delta values, 3 to 12 M
ΔIntima area, mean (mm2)	0.160 [− 0.016 to 0.440]	0.245 [0.004 to 0.780]	− 0.103	(− 0.199 to − 0.007)	0.04
ΔIntima area, max (mm2)	0.362 [− 0.026 to 1.567]	0.590 [0.104 to 1.548]	− 0.286	(− 0.563 to − 0.009)	0.04
ΔMedia area, mean (mm2)	− 0.007 [− 0.112 to 0.081]	− 0.037 [− 0.100 to 0.104]	− 0.015	(− 0.066 to 0.036)	0.56
ΔLumen area, min (mm2)	− 0.610 [− 1.354 to −0.126]	− 0.545 [− 1.168 to − 0.105]	− 0.062	(− 0.226 to 0.102)	0.46
ΔLumen area, mean (mm2)	− 0.859 [− 1.259 to − 0.229]	− 0.830 [− 1.244 to − 0.296]	− 0.115	[− 0.318 to 0.088]	0.27
ΔIntima thickness, mean (mm)	0.019 [0.002 to 0.042]	0.026 [0.006 to 0.067]	− 0.007	(− 0.014 to 0.001)	0.06
ΔIntima thickness, max (mm)	0.080 [0.003 to 0.142]	0.155 [0.036 to 0.303]	− 0.083	(− 0.125 to − 0.042)	< 0.01

Median change from 3 to 12M (delta values) presented with [interquartile range] for each method. β-method coefficients from mixed model analyses are provided with 95%-CI and p-values. For simplicity, the remaining parts of the mixed model equations (constant, random effects and residual errors) are not included. min; minimum, max; maximum

Intraclass Correlation Coefficient (ICC) and Coefficient of Variation (CV) are used for intra- and interobserver analysis. Data analyses were performed using STATA (STATA/IC version 17.0, StataCorp LP, College Station, TX).

Results

Patients

A total of 23 patients with 126 single vessel recordings were available for the present study counting both 3M and 12M pullbacks. Only vessels where both 3M and 12M pullbacks were available for analysis were used. Six pullback pairs were excluded (four due to poor image quality and two with < 10 frames available for analysis). Patient characteristics are summarized in Table 1.

OCT results

The included 114 OCT pullbacks were acquired from 21 LAD, 16 LCX and 20 RCA arteries from 23 patients at 3M and 12M after HTx. A total of 31,228 frames were available from semi-automated analyses and 4999 frames were available from manual analyses.

712 frames from manual analyses had only lumen measurements available due to exclusion of intimal areas dominated by atherosclerotic plaque. These frames were excluded and matching on frame level therefore generated a subset of 4287 frames used for matched analyses.

Inter‑method difference

Inter-method differences from mixed model analyses are specified using slope-coefficients (β).

We observed no significant differences for any lumen measurements in the matched frame analysis, including minimum and mean lumen area (β-method_min = 0.029 mm², p = 0.82; β_mean = 0.080 mm², p = 0.45). The BA-plot showed good agreement for all averages of minimum lumen area (Fig. 2). We found no significant differences between methods for mean or maximum media area (β-method_mean = − 0.019 mm², p = 0.05, β-method_max = 0.049 mm², p = 0.05).

Fig. 2.

Scatter and Bland Altman plots of mean intima area, maximum intima area, minimum lumen area and maximum intima thickness estimated by the manual and semi-automatic method (manual as reference). Mixed model equations are presented with 95%-confidence intervals (CI) and mean differences of Bland Altman plots are presented with 95%-Limits of Agreement (LoA)

Intima thickness estimates were slightly underestimated by the semi-automated software (β-method_mean = − 0.022 mm, p < 0.01; β-method_max = − 0.221 mm, p = < 0.01) (Table 2). Likewise, both mean and maximum intima area were significantly underestimated (β-method_mean = − 0.289 mm², p < 0.01; β-method_max = − 0.695 mm², p < 0.01). BA-plots for both maximum intima area and thickness revealed a systematic increasing semi-automatic underestimation between methods for increasing average estimates.

Table 2.

Mixed effect comparison of semi-automatic & manual estimates using matched frames

Matched frames (4,287)	β-method fixed	95%-CI	p	β-ΔTime (SE) fixed	C fixed	Patient (SE) random	Vessel (SE) random	Residual (SE) random
Layer area (mm2)
Intima area, min	− 0.083	(− 0.17 to 0.01)	0.07	0.184 (0.11)	0.97	0.25 (0.14)	0.18 (0.05)	0.16 (0.09)
Intima area, mean	− 0.289	(− 0.45 to − 0.13)	< 0.01	0.501 (0.08)	1.96	0.90 (0.32)	0.37 (0.11)	0.37 (0.04)
Intima area, max	− 0.695	(− 0.97 to − 0.42)	< 0.01	1.008 (0.28)	3.74	2.36 (1.05)	1.20 (0.49)	1.18 (0.36)
Media area, min	− 0.056	(− 0.08 to − 0.03)	< 0.01	− 0.031 (0.03)	0.76	0.03 (0.02)	0.09 (0.02)	0.02 (0.01)
Media area, mean	− 0.019	(− 0.04 to 0.00)	0.05	− 0.017 (0.03)	1.18	0.07 (0.03)	0.09 (0.02)	0.01 (< 0.01)
Media area, max	0.049	(− 0.01 to 0.10)	0.05	0.036 (0.03)	1.70	0.12 (0.05)	0.09 (0.02)	0.04 (< 0.01)
Lumen area, min	0.029	(− 0.22 to 0.28)	0.82	− 0.784 (0.13)	5.95	0.34 (0.90)	5.22 (1.30)	0.90 (0.10)
Lumen area, mean	0.080	(− 0.13 to 0.29)	0.45	− 0.683 (0.11)	9.08	0.81 (1.00)	5.22 (1.28)	0.64 (0.07)
Lumen area, max	0.180	(− 0.23 to 0.59)	0.39	− 0.330 (0.21)	12.66	2.60 (1.84)	7.20 (1.87)	2.46 (0.30)
Layer thickness (mm)
Intima thickness, min	0.019	(0.01 to 0.02)	< 0.01	0.007 (< 0.01)	0.03	< 0.01 (< 0.01)	< 0.01 (< 0.01)	< 0.01 (< 0.01)
Intima thickness, mean	− 0.022	(− 0.03 to − 0.01)	< 0.01	0.047 (0.01)	0.17	0.01 (< 0.01)	< 0.01 (< 0.01)	< 0.01 (< 0.01)
Intima thickness, max	− 0.221	(− 0.29 to − 0.15)	< 0.01	0.136 (0.02)	0.61	0.03 (0.01)	0.02 (0.01)	0.02 (< 0.01)

Mixed models for individual metrics are presented at each row in the table. The β-coefficients for method are presented with 95%-CI and p-value, using manual method as reference. The β-coefficients for time between 3M and 12M scans (ΔTime) are followed by standard error (SE) using 3M as reference. Random effects are presented with SE for each level in the mixed model. Vessel(SE) can be interpreted as between-vessel within-patient variance. Patient(SE) can be interpreted as between-patient variance, C; Constant, min; minimum, max; maximum, Δ; Delta (change from 3M to 12M)

We did an extensive qualitative re-assessment of the five OCT pullbacks with greatest difference between methods for both mean, minimum, and maximum intima estimates. All were characterized by excessive intimal proliferation, including heterogenous larger segments.

Difference from 3 to 12 M

A significant change from 3 to 12M (ΔTime) was observed using both software methods for all measurements (Table 3). Comparing differences in the change from 3 to 12M by the two methods, we found a significant difference for mean and maximum intima area estimates with the semi-automatic approach showing minor underestimation (β-method_mean, _ΔTime = − 0.103 mm², p = 0.04; β-method_max, _ΔTime = − 0.286 mm², p = 0.04). No significant difference was observed for mean intima thickness (β-method _mean, _ΔTime = − 0.007 mm, p = 0.06) or any lumen or media metrics.

Figure 3 presents scatter and BA-plots of differences from 3 to 12M for mean and maximum intima area as well as minimum lumen area. Change in maximum intima area from 3 to 12M displays increasing variation at higher averages, while the two methods agree well on minimum lumen area at all averages.

Fig. 3.

Change from 3 to 12M (delta parameters) for mean intima area, maximum intima area and minimum lumen area. Depiction in scatter and Bland–Altman plots comparing manual and semi-automated analyses using matched frames (manual as reference)

Including all available frames (31,228) from semi-automated analyses altered results moderately. Intima measurements showed overall better agreement between methods, with maximum intima area now presenting no significant difference (β-method_{max, all frames} = −− 0.199 mm², p = 0.11). 3M to 12M comparison did not change considerably (Table S2).

Descriptive measurements

Descriptive measurements for each method grouped by vessel and 3M or 12M are presented in supplementary tables S5–S7.

Intra‑ and interobserver variation

For the semi-automated methods, intra- and interobserver analyses were based on 10 randomly chosen pairs of 3M and 12M pullbacks, providing 5,624 frames for intraobserver analysis and 5,534 frames for interobserver analysis (Table 4). ICC and CV showed excellent correlations between all area measurements for both intra- and interobserver analysis (ICC = 0.96–1.00; CV = 0.02–0.16). A good correlation was seen for interobserver maximum intima thickness (ICC = 0.87; CV = 0.17). Intra- and interobserver analyses were previously performed for the manual method showing excellent correlations (ICC = 0.98–1.00; CV = 0.01–0.09) [15].

Table 4.

Intraobserver and interobserver variation

Intraobserver variation (5624 frames)	ICC	CV	Mean difference ± SD
Layer area
Intima area, mean	1.00 (95% CI 1.00–1.00)	0.02 (95% CI 0.02–0.03)	0.010 ± 0.028
Intima area, max	0.99 (95% CI 0.97–0.99)	0.07 (95% CI 0.06–0.11)	0.011 ± 0.230
Media area, mean	1.00 (95% CI 0.99–1.00)	0.02 (95% CI 0.01–0.03)	− 0.003 ± 0.019
Lumen area, min	1.00 (95% CI 1.00–1.00)	0.04 (95% CI 0.03–0.06)	− 0.047 ± 0.139
Lumen area, mean	0.99 (95% CI 0.98–1.00)	0.03 (95% CI 0.02–0.04)	0.039 ± 0.250
Layer thickness
Intima thickness, mean	1.00 (95% CI 1.00–1.00)	0.02 (95% CI 0.02–0.03)	0.001 ± 0.002
Intima thickness, max	0.93 (95% CI 0.84–0.97)	0.11 (95% CI 0.08–0.16)	0.004 ± 0.052
Interobserver variation (5534 frames)	ICC	CV	Mean difference ± SD
Layer area
Intima area, mean	0.99 (95% CI 0.98–1.00)	0.06 (95% CI 0.05–0.09)	− 0.006 ± 0.090
Intima area, max	0.96 (95% CI 0.90–0.98)	0.16 (95% CI 0.12–0.23)	0.048 ± 0.438
Media area, mean	0.99 (95% CI 0.98–1.00)	0.03 (95% CI 0.03–0.04)	0.012 ± 0.030
Lumen area, min	1.00 (95% CI 1.00–1.00)	0.04 (95% CI 0.03–0.06)	0.022 ± 0.098
Lumen area, mean	0.99 (95% CI 0.98–1.00)	0.03 (95% CI 0.02–0.05)	0.057 ± 0.302
Layer thickness
Intima thickness, mean	0.99 (95% CI 0.98–1.00)	0.06 (95% CI 0.04–0.08)	− 0.001 ± 0.007
Intima thickness, max	0.87 (95% CI 0.72–0.94)	0.17 (95% CI 0.13–0.26)	0.035 ± 0.076

Intra- and interobserver variation assessed by Intraclass Correlation Coefficient (ICC) and Coefficient of Variation (CV) using matched frames. Mean difference and standard deviation (SD) are listed. Original data used as reference

Average time and frames

An average of 305 ± 101 frames (mean ± SD) per vessel were fully analyzed using the semi-automated software with an estimated time use of 13.9 ± 7.0 min per analysis, based on exact timing of 20 analyses. For matched analysis, we included an average of 38 ± 16 frames per vessel.

Discussion

To the extent of our knowledge, this study is the first to validate a specific semi-automated software for OCT analysis in HTx patients in comparison to a well-established manual approach.

We report the following findings using the semi-automated software in comparison to the manual software for assessment of quantitative OCT measurements:

1. No inter-method differences in lumen and media measurements.

2. Mild underestimation of intima measurements compared to the manual software, predominantly in vessels with extreme intima proliferation.

3. Mild underestimation of intima areas from 3 to 12M after HTx and no difference for mean intima thickness.

4. Semi-automated analyses provide five times more frames using less time.

5. Excellent reproducibility.

Lumen and media measurements

We validate lumen measurements with excellent agreement between methods.

Media area measurements are also well-validated, although we found a small underestimation of minimum media area.

Intima measurements

The uncovered underestimation by the semi-automatic software for intima thickness and area estimates in our study should be taken into consideration. Nevertheless, the numerical intimal underestimation only has minor clinical impact as it primarily concerns patients with severe intimal proliferation, whom are considered high risk patients by both methods regardless. These patients should be followed closely to monitor development over time. For scientific purposes however, the underestimation should be considered before application.

From the qualitative re-assessment of pullbacks with semi-automated intimal underestimation, we found vessels to be characterized by major intimal proliferation. However, using the manual method, such areas were more correctly traced compared to the semi-automatic method, where the outer contours were out of range (Fig. 4). We consider this finding the predominant cause for semi-automated underestimation of intima measurements, supported by the trumpet shaped BA-plots with higher average measurements causing greater differences. The qualitatively reviewed pullbacks constitute the most peripheral data points in the BA-plots of maximum intima area and thickness (Fig. 2).

Fig. 4.

Comparison of cases with different degrees of CAV by angiography and OCT recordings with manual and semi-automatic methods. All images are from 12M examinations. Segmentation borders for OCT-analyses: Red, lumen; purple, internal elastic membrane; green, external elastic membrane

In this study, BA-plots were more consistent for area compared to thickness properties, and in particular for maximum area estimates. Both intima thickness and area display increasing variance at higher estimates, with 12M-measurements being particularly underestimated, predominantly in patients with advanced disease progression.

From 3M to 12M

Most importantly, the two methods detected similar changes from 3 to 12M for all measurements confirming the software applicability for monitoring CAV progression. However, in mixed model analyses we obtained slightly negative β-coefficients for all values due to minor semi-automatic underestimation. Whether the small difference between methods regarding change from 3 to 12M is clinically relevant is uncertain. Including all frames did not alter results considerably.

The boxplot of maximum intima area at 3M and 12M for the three coronaries revealed that semi-automated underestimation predominantly concerns the LAD artery (Fig. 5).

Fig. 5.

Boxplots showing maximum intima area estimated by semi-automatic and manual method for each vessel at 3 months (3M) and 12 months (12M) using matched frames. Median and IQR provided

Less time‑consuming and more analyzed frames

We spent approximately 14 min on a full semi-automated analysis of one vessel. By experience, the manual analysis is much more time consuming—at times, more than an hour. A fivefold increase in the number of included frames while using less time is an additional great advantage. However, the overall accuracy of semi-automatic estimates was higher with matched frame analysis compared to using all frames, likely since validation cannot exceed precision of the original manual analyses. Therefore, the setting of this study does not properly allow us to quantify the gained accuracy of including more frames. We hypothesize including all frames would produce more accurate estimates with the semi-automatic method.

Great reproducibility was seen for both methods in intra- and interobserver analyses [15].

Adding to previous findings

Pazdernik et al. previously revealed the potential of the semi-automatic approach applied to a cohort of 50 patients comprising 100 vessel analyses and a total of 35,600 frames [13]. The group was able to identify significant luminal reduction and intimal thickness progression within the first 12 months after HTx. Nevertheless, the semi-automated software so far has not been validated compared to other well-validated OCT analysis software like QCU-CMS [21].

We confirm the conclusions made by Pazdernik et al. regarding significant progression of intimal thickness and reduction of lumen area. However, and importantly, we add that intima measurements are increasingly underestimated with increasing intimal proliferation. Matched frame analyses were based on vessel regions with no calcifications and atherosclerotic plaques and therefore cannot explain the underestimation.

Specific proposals for improvement

Modifications of the expert guided correction tool, JEI, will allow for more precise correction of intimal borders, thus improving accuracy of analysis for advanced intima proliferation.

Comparing measurements and segmentation analyses to the work of Pazdernik et al., we discovered generally more extensive CAV in our study population [13]. Using more pullbacks characterized by extensive CAV in training of the semi-automatic machine learning algorithm should improve automatic recognition of intima borders in sick patients. The above changes will be integrated to the impending software update in order to optimize assessment of intima measurements.

Clinical implications

The silent clinical presentation of CAV makes early detection difficult without intravascular imaging modalities such as IVUS and OCT [4]. This offline semi-automated OCT-analysis software could potentially increase the revenue from these analyses with even more accuracy and broader application due to economic advantages. In the future, the semi-automatic method could unlock the potential of OCT-analysis as a standardized prognostic marker, spare patients for invasive procedural risks, and lead to earlier diagnoses and better treatment. Early identification of CAV progression encourages starting mTOR treatment, more aggressive reduction of cholesterol and blood pressure, intensified monitoring of diabetes, etc. [12, 23]. A prospective study is warranted for quantification of specific diagnostic cut-off points for CAV using OCT measurements. This could potentially help design an OCT-based CAV-burden classification system.

Study limitations

The OCT image data from this study origin from a single transplant center. However, the evaluation has been meticulous, and the numbers of vessels and patients compared are high.

The semi-automated algorithm was not trained on patients with the same extent of CAV as patients from our study population.

The semi-automated and manual analyses were performed by two different expert operators with three years in between. Nevertheless, interobserver analyses were sound for both methods.

No exact timing data were available for manual analyses, however experts estimate approximately one hour was used for a full manual analysis of one vessel.

Conclusion

In validation of the semi-automatic software for OCT analysis in HTx patients, we report excellent validation of lumen and media area measurements, while intima layer metrics were underestimated increasingly with intimal proliferation. Intimal progression from 3 to 12M was slightly underestimated by the semi-automated OCT software with unknown clinical relevance. The semi-automated software has the future potential to provide robust and time-saving evaluation of CAV progression.

Abbreviations

BA

Bland–Altman

CAV

Cardiac allograft vasculopathy

CI

Confidence Interval

CV

Coefficient of Variation

HTx

Heart Transplantation

ICC

Intraclass Correlation Coefficient

ICWA-OCT

Iowa Coronary Wall Analyzer for OCT

IVUS

Intravascular Ultrasound

IQR

Inter Quartile Range

JEI

Just-Enough-Interaction

OCT

Optical Coherence Tomography

Data availability

The data underlying this article cannot be shared publicly due to the privacy of individuals that participated in the study. The data will be shared on reasonable request to the corresponding author.