A Mechanical Approach for Smooth Surface Fitting to Delineate Vessel Walls in Optical Coherence Tomography Images

Abstract

Automated analysis of vascular imaging techniques is limited by the inability to precisely determine arterial borders. Intravascular optical coherence tomography (OCT) offers unprecedented detail of artery wall structure and composition, but does not provide consistent visibility of the outer border of the vessel due to limited penetration depth. Existing interpolation and surface fitting methods prove insufficient to accurately fill the gaps between the irregularly-spaced and sometimes unreliably identified visible segments of the vessel outer border. This paper describes an intuitive, efficient, and flexible new method of three dimensional surface fitting and smoothing suitable for this task. An anisotropic linear-elastic mesh is fit to irregularly-spaced and uncertain data points corresponding to visible segments of vessel borders, enabling the fully-automated delineation of the entire inner and outer borders of diseased vessels in OCT images for the first time. In a clinical dataset, the proposed smooth surface fitting approach had great agreement when compared to human annotations: areas differed by just 11±11% (0.93±0.84 mm²), with a coefficient of determination of 0.89. Overlapping and nonoverlapping area ratios were 0.91 and 0.18, respectively, with sensitivity of 90.8 and specificity of 99.0. This spring mesh method of contour fitting significantly outperformed all alternative surface fitting and interpolation approaches tested. The application of this promising proposed method is expected to enhance clinical intervention and translational research using OCT.

I. Introduction

Coronary artery disease (CAD) is the scourge of modern society [1], responsible for 1 in every 7 deaths in the United States [2]. Imaging of coronary vessels is a primary step in prognosis and treatment of CAD, and various imaging modalities have been introduced into the clinic [3]. Invasive techniques are the most effective modalities for detecting the vessels’ borders and the presence of atheromatous plaques, characterizing their extent and composition [4]. Optical coherence tomography (OCT) utilizes light to perform imaging, leading to remarkable resolution (10–20 μm axially, 20–40 μm laterally [5]) which allows for highly detailed imaging of the near field. However, the use of light results in relatively low penetration depth (1–2.5 mm [5]) with strong dependence on tissue optical properties, allowing visual identification of vessel outer border (intima- or plaque-media interface) only in healthy or minimally-diseased vessel segments [6]. Despite this shortcoming, OCT has been proven to be not just feasible and safe for clinical use [7], [8], but has also demonstrated the capacity to offer insights and detailed information not offered by other imaging modalities [8]–[10]. OCT can diagnose CAD [6], characterize the majority of plaque types with excellent reproducibility [11], [12], guide coronary interventions such as stent deployment [7], and assess stents after deployment [11], [13]. Nonetheless, the aforementioned limitation for outer border identification prevents adequate estimation of plaque burden and proper vessel sizing, both being clinically relevant data.

The growing evidence of OCT’s value in clinical and research arenas has increased demand to measure plaque burden and content. Yet, it is precisely aspects of the plaque that limit penetration (e.g. lipid pools), and automatic and accurate approximation of the full vessel outer border is challenging with OCT, especially when only a subset of the feature is visible. While some advancements in automated OCT analysis have worked within this constraint to segment and quantify superficial plaque [14], [15], others have sought to overcome the limitation. A preliminary approach towards automatic delineation of the wall utilized standard filtering and edge detection methods, coupled with frame-by-frame shape fitting, to approximate the outer border and vessel cross-sectional area [16]. Later work with similar aims performed automatic multilayer segmentation of the vessel wall using a method akin to work in retinal OCT layer segmentation: a front propagation scheme relying on grayscale gradient information [17]. However, this approach was limited by validation only in healthy wall regions, where the absence of significant plaque permitted clear visibility of the various layers. The challenge at hand therefore remained unresolved for clinically-relevant applications.

Physiological understanding and constraints can be brought to bear on the problem. Plaque burden follows a continuity inside the vessel and does not change dramatically from frame to frame. Therefore, using a priori knowledge of the structural and geometric constraints of vessel walls and working in 3D space, rather than on a frame-by-frame basis, the limited visibility of the outer vessel wall can be largely overcome. Indeed, the non-visible parts of the outer vessel border can be approximated in the 3D space to estimate the full arterial wall position.

Many methods exist for filling gaps in three-dimensional (3D) information, though most are variations of interpolation or surface fitting which are limited in flexibility, intuitiveness, ability to incorporate secondary information, and computational efficiency. Interpolation methods construct functions that intersect known data points or positions, and utilize various methods to approximate the values or positions lying between them [18], [19]. Fundamentally, smoothing is limited in interpolation methods, as functions are constrained to pass through available data points, which may themselves be non-smooth and/or unreliable. When the available data may include noise, error, or inaccuracies, or are not reliable, as is the case with the application at hand, surface fitting is generally acknowledged to be preferable to interpolation; scattered data fitting techniques seek to find a reasonably-approximating, ostensibly-smooth surface to represent the underlying data trend [18]–[20]. However, the shapes and contours that can be achieved with existing surface fitting techniques are fundamentally constrained—most surfaces must conform, locally or globally, to a polynomial function. This presents a vital limitation, particularly in applications to natural and complex systems which cannot be represented with sufficient fidelity by such simplified expressions. Furthermore, because neither standard interpolation nor surface fitting techniques can incorporate additional inputs from the rich information available in imaging data, resulting approximations are heavily biased when the input data points are not representative of, or equally-distributed from, the underlying data. In the present case of the vessel wall, we expect these techniques to underestimate unknown values of wall position because missing data points (non-visible wall segments) are inherently more likely to have values (radial depths) at the extremes of, or exceeding, those of the available data point values (visible wall segment depths).

The approach described here leverages key benefits of simple mechanical systems. For example, the simplicity and elegance of Hookean linear-elastic spring models lend themselves to unparalleled efficiency because they allow straightforward expression in matrix operations and the associated benefits in solving systems of equations. For this reason, deformable mass spring tissue models have been implemented instead of finite element tissue models to realistically simulate interactive surgeries in real time [21], and a spring surface mesh has been implemented to simulate dynamic organ behavior corresponding to surgical operations with precision and speed [22]. While not applied to image processing, these applications exemplify the efficiency of Hookean linear-elastic springs models where large systems of equations must be constructed and solved to calculate three-dimensional response to external loads. Nevertheless, application of mechanical models to the field of image processing has been very limited. Bayesian statistics and statistical mechanics have been applied to probabilistic image restoration techniques and extended to additional image processing tasks, such as edge detection, image segmentation, motion detection, and image compression [23]. However, more dynamic systems like the one implemented in this work have not been explored.

In this work, we present a promising new smoothing surface fitting method that overcomes the challenges of existing interpolation and surface fitting methods, and enables the delineation of the entire inner and outer borders of vessels in OCT images. Building upon our previous work [16], the new method, which is inspired by intuitive linear-elastic springs common in classical (Newtonian) mechanics, is both efficient in 3D space and conspicuously effective. The novelty of this work lies not only in its application—in autonomous delineation of the full outer border of a diseased coronary artery using OCT imaging alone—but also in the approach used to reconstruct and smooth surfaces using incomplete information. The algorithm is rigorously validated in 7 patients (724 frames) through comparison to expert annotations, and its performance is compared to alternative interpolation and surface fitting approaches.

II. Materials and Methods

Our method uses the following steps (Fig. 1): 1) the inner (lumen) border of the vessel is detected in each frame and smoothed as a surface; 2) the image is flattened relative to the lumen and visible edges of the outer wall are identified using image contrast enhancement, 3D filtering, application-specific edge-detection kernels, and region-growing; 3) an anisotropic linear-elastic mesh is fit to the identified edges with applied forces to generate a continuous, smooth surface along the outer border; 4) the results are finally reconstituted to the pre flattened reference frame and converted to a Cartesian coordinate system. Note that the surface fitting method was used twice; it was used first to smooth the inner border (for which no information was missing but the available information was a bit noisy and not perfectly reliable), and subsequently to both complete and smooth the outer border (when only incomplete, irregularly-spaced, and sometimes unreliable information was available).

Fig. 1.

Major methodological procedures include lumen detection and smoothing, visible outer border segment detection, surface fit to wall segments, and surface reconstitution with conversion from cylindrical to Cartesian coordinates.

A. General Description: Spring Mesh Surface Construction

We define, as for any surface fitting approach [20], a finite set of m known points P_r ∈ ℝ³ and tolerances ϵ_r ∈ ℝ₊. The desired surface S ⊂ ℝ³ approximates the positions of the known (and unknown) points such that

$$\Vert {\mathit{S}}_{{\mathit{P}}_r}-{\mathit{P}}_r\Vert \le {ϵ}_r,r\in \{1,\dots ,m\}.$$ (1)

Given the application to image processing and the constraints of implementation, the assumption is made that known points can be assigned a discretized position at an indexed location. Specifically, here we describe implementation in a 3D discretized space with q sequential planes j∈[1;q] each containing p columns i∈[1;p], where each position can be assigned an index of (i,j,k).

The global method is based upon a virtual anisotropic linear-elastic mesh of interconnected nodes and springs (Fig. 2). The aforementioned 3D discretized space, with an (i,j) index assigned to each distinct column, contains a single node which is constrained to that indexed column but free to translate in the $\widehat{k}$-direction. This node represents the position of the fit surface at the given index, S_ij. Springs join adjacent nodes in the $\widehat{\iota }$- and $\widehat{j}$-directions. If there exists a known point P_r in column (i,j)—P_ij—a position-fixed anchor may be placed at the location of P_ij. Stated otherwise, an anchor may be placed at (i,j,k), where index k is assigned the value of P_ij—p_ij. This anchor provides a set non-zero reference—a mechanical equivalent to a voltage source from ground. A base layer (L_ij) of variable height (l_ij) may also be present, and represents a nominal, average, or expected position. Springs oriented in the $\widehat{k}$-direction connect the nodes either to anchors or, if not present in the given column, the base layer. Each class of springs—those adjoining nodes 1) within the plane j (i to i+1 and i-1), 2) between adjacent planes (j to j+1 and j-1), 3) to anchor points (P_ij), and 4) to the base layer (L_ij)—may be of different stiffnesses (K₁, K₂, K₃, and K₄, respectively, where K is the spring constant). We note here that any tolerance condition (1) is guaranteed to be met if anchored springs (class three) are infinitely stiff, i.e. K₃=∞, though this comes at a cost to smoothness as the node position is absolute and fixed to coincide with the anchor. The stiffness of each spring could also be, though is not here, assigned individually (for example, such that $K_3^{ij}$ is proportional to confidence in the anchor point position p_ij, or other approaches akin to a weighting function for smoothing weighted regression [24]), but this would increase complexity of the system to an extent not deemed worthwhile in many cases given the additional variable features described next.

Fig. 2.

Schematic showing system structure in two and three dimensions. Spring colors indicate class with different stiffness K: 1) blue (K₁); 2) yellow (K₂); 3) red (K₃); and 4) green (K₄). Floating black blocks indicate anchor points (P_ij) coinciding with known data points intersecting the column. A light-gray base layer is visible along the bottom (shown here as constant height, l_ij=l). Semi-transparent gray arrows indicate forces applied to each node (F_ij).

A unique force, F_ij, can be applied to each node. The magnitude and direction of this force may be calculated from original data, spatial position of the column, or any other desired source. This ability to incorporate additional information provides excellent flexibility in application. For example, in the application presented here, edges were less likely to be detected with increasing distance (displacement in $\widehat{k}$-direction). Therefore, a spring mesh alone should underestimate the surface between the detected segments. However, by leveraging this a priori knowledge, forces can be applied that “inflate” the mesh outwards in regions in which no anchoring surface segments are detected.

With this structure in place, a system of linear equations can be derived that unambiguously solves for the position of each node. This system is solved for its equilibrium state, a condition described as:

$$\sum \mathit{F}=\mathbf{0}.$$ (2)

Because each node is necessarily constrained within its column:

$$\sum F_i\equiv 0,\sum F_j\equiv 0.$$ (3)

It therefore only remains to solve the balance of forces in the $\widehat{k}$-direction. We begin by assuming a common, generic Hookean linear-elastic spring model, in which restorative spring force (F_spring) is linearly proportional to its displacement (X) by a spring coefficient (K) defining its stiffness [25]:

$$F_{spring}=-KX.$$ (4)

With the model defined, the system of equations can be derived. Defining the initial lengths of springs to be 0, we combine forces exerted by the various springs on each node in the $\widehat{k}$-direction:

$$\sum F_k^{ij}=K_1{d}_{i-}\text{sin}{\theta }_{i-}+K_1{d}_{i+}\text{sin}{\theta }_{i+}+K_2{d}_{j-}\text{sin}{\theta }_{j-}+K_2{d}_{j+}\text{sin}{\theta }_{j+}+F_{ij}^{*}+F_{ij}$$ (5)

$$\overline{F_{lj}^{*}}=\{\begin{array}{l}{K}_3\left(p_{ij}-S_{ij}\right)\widehat{k}\text{if}\exists P_{ij}\hfill \\ K_4\left(l_{ij}-S_{ij}\right)\widehat{k}\text{if}\cancel{\exists }{P}_{ij}\hfill \end{array}$$ (6)

where d and θ are, respectively, the distance and angle between adjacent nodes, i and j of the subscript indicate the direction of adjacency (among or between planes, respectively), and +/− signs of the subscript indicate direction of adjacency (increasing or decreasing node number). As before, p_ij is the height of the anchor point (tentative surface segment) in column (i,j), and S_ij is the position of the node in the column, or alternatively, the position of the fit surface at index (i,j). Note that as the sine of angle θ is inversely proportional to the spring displacement d,

$$\text{sin}{\theta }_{i\pm }=\frac{S_{(i\pm 1)j}-S_{ij}}{d_{i\pm }},\text{sin}{\theta }_{j\pm }=\frac{S_{i(j\pm 1)}-S_{ij}}{d_{j\pm }},$$ (7)

the above relationship (5) can be simplified to:

$$\sum F_k^{ij}=K_1\left(S_{(i+1)j}-S_{ij}\right)+K_1\left(S_{(i-1)j}-S_{ij}\right)+K_2\left(S_{i(j+1)}-S_{ij}\right)+K_2\left(S_{i(j-1)}-S_{ij}\right)+F_{ij}^{*}+F_{ij}=0.$$ (8)

Rearrangement, grouping, and normalization by K₁ to reduce the number of parameters yields:

$$0=-\left(2+2\frac{K_2}{K_1}\right)S_{ij}+S_{(i+1)j}+S_{(i-1)j}+\frac{K_2}{K_1}{S}_{i(j+1)}+\frac{K_2}{K_1}{S}_{i(j-1)}+\frac{F_{ij}}{K_1}+\{\begin{array}{l}\frac{K_3}{K_1}\left(p_{ij}-S_{ij}\right)\text{if}\exists P_{ij}\hfill \\ \frac{K_4}{K_1}\left(l_{ij}-S_{ij}\right)\text{if}\cancel{\exists }{P}_{ij}\hfill \end{array}.$$ (9)

If all N nodes are numbered sequentially, such that (i,j) corresponds to n and (i,j+1) corresponds to (n+p), and N is equal to p×q, the full system of equations can be expressed as Ax=b, where A is a sparse N×N square matrix:

$$A_{nn}=-2-2\frac{K_2}{K_1}-\{\begin{array}{l}\frac{K_3}{K_1}\text{if}\exists P_{ij}\hfill \\ \frac{K_4}{K_1}\text{if}\nexists P_{ij}\hfill \end{array};$$ (10)

$$A_{n(n\pm 1)}=1;$$ (11)

$$A_{n(n\pm p)}=\frac{K_2}{K_1}.$$ (12)

Furthermore, x is a vector of node positions:

$$\mathit{x}=\left[\begin{array}{c}{S}_1\\ ⋮\\ S_N\end{array}\right].$$ (13)

Finally, b is a vector with the following entries:

$$b_n=-\frac{F_n}{K_1}-\{\begin{array}{l}\frac{K_3}{K_1}{p}_{ij}\text{if}\exists P_{ij}\hfill \\ \frac{K_4}{K_1}{l}_{ij}\text{if}\nexists P_{ij}\hfill \end{array}.$$ (14)

Multiplying both sides of the equation by the inverse of matrix A, the position of all N nodes can be solved simultaneously with a single operation with computation complexity O(N^1.5):

$$\mathit{x}={\mathit{A}}^{-1}\mathit{b}.$$ (15)

This approach may be used in Cartesian coordinates for a non-folded surface or, as in the present application, a cylindrical surface expressed in cylindrical (polar) coordinates. When working in cylindrical coordinates, edge continuity is achieved by connecting nodes at the start and end of each plane j, as shown with the dashed-line springs (Fig. 2). Other boundary conditions can also be readily enforced.

As described in the following sections, this methodological approach for constructing a continuous surface from incomplete information was applied in OCT to 1) smooth a contour in which tentative locations of the entire surface are known, and 2) construct a surface when only limited fragments of the surface have been tentatively located.

B. Application: OCT Wall Detection

The spring mesh surface construction method was applied to automatically and fully delineate the borders of a diseased vessel in OCT images—to our knowledge the first method to achieve this feat. The approach was implemented using the MATLAB software (MATLAB R2017a, The MathWorks, Inc., Natick, MA, USA).

1). Lumen Detection & Smoothing

Initial detection of the lumen followed the procedure used by Athanasiou et al. [12]. In brief, image data were converted to binary format with Otsu’s thresholding method, common artifacts (i.e. catheter sheath and guidewire backscattering) were purged, and the most proximal point in each column (nonzero point of lowest radial distance) was retained. Simple two-dimensional (2D) smoothing was performed on these points, and they were then connected through linear interpolation to points in laterally-adjacent columns with straight lines, resulting in a continuous contour within each frame.

While minimally smoothed, the resulting lumen surface was not necessarily continuous in three dimensions (longitudinally). To further smooth the lumen as a continuous 3D surface, an anisotropic linear-elastic mesh was fit to the points using the approach described earlier (II.A). In this application, each column contained an anchor point (P_ij), which eliminated the need for parameters defining the base surface (L_ij) and the spring coefficient for connections between nodes and this surface (K₄). No force function was applied in this scenario, eliminating a third parameter (F_ij). The remaining parameters, K₂/K₁ and K₃/K₁, were determined by minimizing the sum of radial distance magnitudes between the node equilibrium positions and the validation annotations of the lumen:

$$\underset{{}_{\frac{K_2}{K_1},\frac{K_3}{K_1}\ge 0}}{\text{min}}{\sum }_{r=1}^N\Vert \mathit{S}{\left(\frac{K_2}{K_1},\frac{K_3}{K_1},{\mathit{P}}_r\right)}_{{\mathit{P}}_r^{*}}-{\mathit{P}}_r^{*}\Vert$$ (16)

where P* are the validation annotation positions. (See section II.C, “Validation,” for more information on these annotations.) Minimization was achieved by a constrained optimization nonlinear programming solver employing the interior-point algorithm. Each iteration step was calculated through factorization, and forward finite differences were used to estimate the objective gradients; the Broyden-Fletcher-Goldfarb-Shanno algorithm was employed to approximate the Hessian matrix. The iterative process terminated when changes in parameters decreased beyond 10⁻¹⁰.

2). Outer Border Segment Detection

Identification of the outer border candidate edges drew upon image enhancement and edge detection approaches specifically adapted for this application, as described in detail in the supplemental materials. In brief, polar images were first flattened relative to the lumen to better horizontally align layers of the same radial depth, improve filtering efficacy, and ease layer boundary identification through improved shape consistency [26], [27]. Contrast enhancement and image compensation [28] were performed on the flattened image, speckle noise was reduced through 3D median filtering [29], Sobel-Feldman edge-detection kernels detected grayscale gradients, and skewed distribution kernels—specially designed to account for the nominal size of the media layer—were utilized in convolution to identify the radial light-dark-light intensity transition indicative of the media layer [6], the inner surface of which was considered here as the outer border of the vessel.

3). Surface Fitting

Given border segments detected in 3D space, it was necessary to fill gaps between detected segments and ensure continuity from frame to frame. Continuity and smoothness are inherently observed in nature, and cross-sectional area does not vary considerably in adjacent frames. It was therefore necessary to fit a smoothed surface with the available information of tentative partial surface location. This was accomplished using the novel anisotropic linear-elastic mesh surface fitting method introduced earlier (II.A, Fig. 3).

Fig. 3.

Diagram illustrating how the surface fitting method was implemented in this application within each frame (top) and between frames (bottom). The detected (candidate) edges served as anchor points, while the proximal side of the flattened image (i.e. the lumen) acted as the base layer. Each uniformly-spaced node was constrained to displacement in the radial ($\widehat{k}$) direction, and was acted upon by an outward force proportional to the sum of column intensity values. Each node was connected to the nearest nodes in each direction, and the nodes on either end of a frame were connected to ensure continuity in cylindrical coordinates. Note that, while 15 nodes are illustrated here for clarity, each of the 504 columns in every frame contained a node. Schematic representation and color correspondence as in Fig. 2.

In this application, only columns in which border segments were detected include an anchor point (P_ij). It was observed that base surface position could be uniformly set to 0 (l_ij=0) with minimal impact on performance, so this was used to decrease complexity and reduce arbitrary offsets. Here, a force function was selected to apply a unique force to each node that incorporated information from the original imaging information:

$$F_{ij}=\alpha \sum I(i,j)$$ (17)

where α is a scaling coefficient, I is the pixel intensity of the original image, and as before, F_ij is node-specific force applied to the mesh. Sum of pixel intensity was utilized because thick, high-intensity regions often indicate the presence of a thick, diseased intima, which corresponds to greater outer border depth. This scheme also prescribes near-zero force in regions obscured by the guidewire shadow, a desirable feature resulting in smooth interpolation-like continuity with adjacent regions.

While multiple force functions were tested, including those proportional to lumen cross-sectional area or simply a constant, the sum of intensities in each column yielded the best result. The non-fixed parameters for this system, K₂/K₁, K₃/K₁, K₄/K₁, and α/K₁, were determined using the aforementioned constrained optimization nonlinear programming solver by minimizing the sum of radial distance magnitudes between the node equilibrium positions and the validation annotations of the outer border:

$$\underset{\frac{K_2}{K_1},\frac{K_3}{K_1},\frac{K_4}{K_1},\frac{\alpha }{K_1}\ge 0}{\text{min}}{\sum }_{r=1}^N\Vert \mathit{S}{\left(\frac{K_2}{K_1},\frac{K_3}{K_1},\frac{K_4}{K_1},\frac{\alpha }{K_1},I,{\mathit{P}}_r\right)}_{{\mathit{P}}_r^{*}}-{\mathit{P}}_r^{*}\Vert$$ (18)

where P* are the validation annotation positions. (See section II.C, “Validation,” for more information on these annotations.)

Interpolation and surface-fitting to the detected surface segments were also conducted with existing methods for performance comparison.

4). Surface Reconstitution & Coordinate Conversion

After the equilibrium node positions were calculated, the resulting surface was reconstituted to the non-flattened state by shifting each column distally by the radial distance of the lumen in the original image (reversing the original flattening process; see section II.B.2, “Outer Border Segment Detection”). The reconstituted surface was finally converted from the cylindrical to Cartesian coordinate system—transforming the open surface into a wrapped cylindrical surface—using the standard transformation.

C. Validation

The autonomous border detection method was validated through direct comparison to manual annotations made by expert interventional cardiologists in the same OCT clinical dataset. An average border annotation of two experts was determined by taking the mean radial distance of both annotations from the center of the image around the perimeter in each frame (as in [12]). Annotations were compared in 724 frames from 7 unique pullbacks (series of images) acquired from the same number of unique patients.

The detected borders achieved with the presented methodology were then overlaid on the average OCT border annotations (considered ground truth), and the following parameters were calculated:

$$\text{Sensitivity}=\frac{TP}{TP+FN}\times 100={\text{R}}_{\text{over}}\times 100;$$ (19)

$$\text{Specificity}=\frac{TN}{FP+TN}\times 100;$$ (20)

$$\text{Jaccard Index}=\frac{TP}{TP+FP+FN}\times 100;$$ (21)

$$\text{Dice Index}=\frac{2TP}{2TP+FP+FN}\times 100;$$ (22)

$${\text{R}}_{\text{nonover}}=\frac{FN+FP}{TP+FN}$$ (23)

where TP are true positives, TN are true negatives, FP are false positives, and FN are false negatives. Here, TP are considered those pixels enclosed by both the algorithm results and expert annotations, TN are those pixels of the full 500×500 pixel image enclosed by neither annotation, FP are those pixels enclosed by only the algorithm result, and FN are those pixels enclosed by only the expert annotation. The ratios of overlapping (R_over) and non-overlapping (R_nonover) areas between the algorithm results and expert annotations ((19) and (23), respectively) were defined similar to Athanasiou et al. [12] to represent sensitivity and specificity of automated image segmentation in a way that is not skewed and inflated by the image size, particularly when the region of interest is only a subset of the actual image (as in OCT). The distribution of the absolute radial distance between the expert annotations and algorithm results was also determined, the coefficient of determination (R²) was calculated for the resulting cross-sectional areas, and Bland-Altman analysis of the delineated cross-sectional areas was performed.

The autonomous border detection method was further validated through indirect comparison to the manual annotations made by an expert interventional cardiologist in a corresponding intravascular ultrasound (IVUS) clinical dataset. IVUS, which has lower resolution but greater penetration depth than OCT and is well validated for identification of vessel outer wall, was treated as the gold standard. However, evaluation criteria for validating between modalities is limited—only differences between areas enclosed by the annotation results and expert tracings and their R² could be calculated, and Bland-Altman analysis of the areas was performed.

D. Dataset

Seven patients who underwent invasive angiography and had one or more coronary stenoses of at least intermediate severity were included in the study. The study was approved by the Ethics Committee of the institution and all patients gave informed consent.

Angiography was performed following a standard protocol, using radial or femoral access. After the diagnostic angiography, 200 μg of intracoronary nitroglycerin and 5,000 IU of unfractionated heparin were infused. Investigators used the C7-XR FD-OCT optical frequency domain intravascular imaging system and the DragonFly catheter (St. Jude Medical, Lightlab Imaging Inc., Westford, MA, USA). This imaging system offers a maximum frame rate of 100 frames per second, 500 lines per frame, a scan diameter of 10 mm, and axial resolution of 15 μm. The monorail imaging catheter was advanced distal to the target lesion. The automated pullback was performed at 20 mm/s (system nominal pullback speed) while the blood was removed by a short injection of iso-osmolar contrast at 37°C through the guiding catheter. The optimal volume/time intracoronary infusion of contrast was tested to achieve a complete blood clearance in the vessel lumen.

Per protocol, IVUS imaging of the coronary segments imaged with OCT was also conducted. The IVUS catheter, a Revolution 45 MHz rotational IVUS imaging catheter (Volcano Inc., Rancho Cordova, CA, USA) was placed at the same distal position as the OCT catheter, using as reference a fiduciary anatomical point such as a bifurcation or a calcification spot. An angiographic acquisition was obtained with the imaging catheter in the distal position to confirm matching points for the IVUS and OCT starting runs. An automated motorized pullback at 0.5 mm/s was performed.

Images were digitally stored for off-line analysis, and all imaging data sets were completely anonymized and transferred to the core lab for analysis. For the purposes of this study, OCT datasets in which blood removal was found post hoc to be grossly insufficient were not utilized. Delimiting segments of useable pullbacks captured prior to or following injection of contrast were similarly excluded. To ensure the same vessel segments were analyzed with both techniques, matching of the OCT and IVUS pullbacks was performed using landmarks such as side branches and other anatomical features, thereby providing frame-by-frame correspondence. Experts were thereafter blinded to the correspondence of imaging files.

III. Results

A. General Performance

As the method is 3D and global for a given pullback, run time was positively, but non-linearly, related to the number of frames in each analyzed pullback. Average execution times for the major steps—executed on a machine running Intel Xeon processors with 12 cores (2.80 GHz) and 12 GB RAM—are reported in Table I. Execution time is dominated by initial lumen and visible outer border segment detection, while the lumen smoothing and surface fitting steps are exceptionally fast (2.74±0.28 ms and 40.2±7.5 ms per frame, respectively). Conversion from polar to Cartesian coordinates, as executed by the authors, is also a significant contributor to the run time (contribution not shown). The one-time optimization of the inner and outer border surface fit parameters took 7.8 and 10.1 minutes, respectively, to execute.

TABLE I.

Run Times for Major Steps

Step	Average Time per Frame (±SD)
Initial Lumen Detection	4.2±1.5
Lumen Smoothing	0.00274±0.00028
Image Flattening	0.0064±0.0013
Edge Detection	5.35±0.85
Surface Fittinga	0.0402±0.0075
Setup	0.0363±0.0070
Solve	0.0039±0.0013

Times are expressed in seconds as average and standard deviation (SD) across pullbacks.

^aThe “Surface Fitting” step total is presented along with its two contributing sub-steps (i.e. the italicized times do not represent additional run time)

As a consequence of the patient inclusion criteria that sought high plaque burden vessels, the outer wall was often poorly visualized. While the experts were able to identify sufficient segments of the outer border to confidently estimate the full border in most frames, the edge detection process presented here identified segments of the outer border in only 57% of columns (A-lines). It is noteworthy that these sparse detected edges are the anchor points (P_r) for the subsequent surface fit.

Optimization was tested for stability with respect to the initial value to ensure robustness and identification of global minimum.¹ For the more challenging outer border, the optimization was stable over 7 orders of magnitude in the range of initial conditions, and the optimal output identified over 20 orders of magnitude of initial value was produced within this stable range (Supplemental Fig. 2). The process was also tested for convergence with respect to the number of frames used in the optimization. As seen in the performance curves (Fig. 4a and Supplemental Fig. 3), all performance metrics improved with increasing numbers of frames included in the optimization, stabilizing around 30 frames (meaning that performance improved only slightly when the number of frames used for optimization was increased from 30 to the full 724). Output parameter values largely converged in the same manner (Fig. 4b). Again noteworthy are not only convergence as frame numbers increase, but profound reduction in error and variability (which on the logarithmic scale of Fig. 4b appear to extend infinitely in the negative direction at high values). When provided with the same inputs, the optimization algorithm consistently arrived at the same results. Similar testing was performed for the less complex lumen smoothing operation optimization, which demonstrated even greater stability (results not shown). Having shown method stability and convergence, all frames were used in the optimization with initial values for all parameters set to 10⁻³, which produced optimal results during the initial condition sensitivity testing. Final optimal parameters for the inner (lumen) border surface fit were: K₂/K₁=6.74×10⁻³; and K₃/K₁=2.02×10⁻². For the outer border surface fit, the final optimal parameters were: K₂/K₁=1.09×10⁻²; K₃/K₁=3.92×10⁻³; K₄/K₁=1.10×10⁻⁴; and α/K₁=1.36×10⁻⁶.

Fig. 4.

Test of (outer border) optimization convergence as the number of frames used in the optimization was increased. (a) Performance metrics (sensitivity, specificity, Jaccard Index, Dice Index, R_over, and R_nonover); (b) Normalized parameter output values (K₂/K₁, K₃/K₁, K₄/K₁, α/K₁).

B. Lumen Detection

The areas delineated by the inner (lumen) border results were first assessed. Here, the initial position of the lumen was determined for each column, and the surface-fitting approach presented above was used just for its smoothing capability. Fig. 5 shows the error and Bland-Altman plots comparing the algorithm to the average expert annotations. Direct comparison of overlapping regions delineated by the human expert annotations and algorithm results in OCT were made as outlined in the “Validation” section (II.C). The original lumen trace differed from that of the human expert by 0.12±0.43 mm (average ± standard deviation; 0.01±0.44 mm overall, i.e. not absolute distance); the smoothed lumen trace differed by 0.11±0.35 mm, on average (0.00±0.36 mm overall). Sensitivity, specificity, Jaccard index, Dice index, R_over, and R_nonover parameters are provided for the lumen before and after the smoothing surface-fit operation (Table II). The mean and median percent change in frame-to-frame lumen area of the final smoothed lumen contours (7.5% and 4.4%, respectively) were between that of the human experts (6.0% and 3.8%, respectively) and that of the original lumen contours (9.3% and 5.0%, respectively).

Fig. 5.

Comparison of algorithm detection and human expert annotation of the lumen. (a) Error plot showing strong correlation between the two areas (R² = 0.94 before smoothing, 0.96 after smoothing; dashed line shows 1-to-1 ratio); (b) Bland-Altman plot (average: −0.36 mm² before smoothing, −0.20 mm² after smoothing; standard deviation: 0.82 mm² before smoothing, 0.65 mm² after smoothing) illustrates no substantial systematic error (error is distributed).

TABLE II.

Performance Metrics Comparing Regions Delineated by Inner Border as Originally Detected and After Smoothing Surface Fit

Metric	Original	Smoothed (Surface Fit)
Sensitivity	98.46	97.54
Specificity	99.47	99.59
Jaccard Index	91.51	92.16
Dice Index	95.57	95.92
Rover	0.9846	0.9754
Rnonover	0.0914	0.0830

Inner Border = Lumen Border

C. Outer Border Detection

The full area delineated by the outer border as determined by experts and the algorithm (Fig. 6) was also assessed.² The outer border was determined with the surface-fitting method presented here using only partial detected contour segments (Fig. 7). In directly comparing the tracing of the outer border made by the algorithm to that of the human experts (Supplemental Fig. 4), the mean difference between the tracing radial depths was 0.09 mm with a standard deviation (SD) of 0.53 mm (0.33 mm with a SD of 0.43 mm for the absolute difference/distance).

Fig. 6.

Two examples of fully delineated frames. Green tracings represent the expert annotation, while magenta tracings depict the algorithm result. Note that the majority of the outer border is not visible in the second frame.

Fig. 7.

Outer border surface as detected by the algorithm in one patient. Images illustrate bounding slices of the flattened image volume. The magenta surface represents the algorithm result; cyan segments represent the detected outer border edge segments, which were used to calculate the surface. Note that the segments tend to appear closer to the lumen surface, where penetration depth has less of an impact on visibility, but the surface contour is not limited to this superficial region.

The average absolute (net) difference area between the algorithm results and human annotations in each frame was 0.93 mm² (10.9%; overall mean of 0.07 mm²) with a SD of 0.84 mm² (11.0%; 1.25 mm² for overall difference); the algorithm provided a very slight overestimation of the cross sectional area. Fig. 8 shows the error and Bland-Altman plots comparing the areas determined by the algorithm to those delineated by the average expert annotations. Direct comparison of overlapping regions enclosed by the annotated outer border is quantified in Table III.

Fig. 8.

Comparison of algorithm results and human expert annotations of the outer border. (a) Error plot showing strong correlation between the two areas (R² = 0.89; dashed line shows 1-to-1 ratio); (b) Bland-Altman plot (average: 0.07 mm², standard deviation: 1.25 mm²) illustrates no clear systematic error.

TABLE III.

Performance Metrics Comparing Regions Delineated by Outer Border as Detected by Various Interpolation & Surface Fitting Techniques

Metric	Anisotropic Linear-Elastic Mesh	Interpolation Methods			Surface Fitting Methods
		Linear Interpolation	Nearest Neighbor Interpolation	Cubic Spline Interpolation	Polynomial Surface Model (5th Degree)a	Local Linear Regression	Local Quadratic Regression
Sensitivity	90.79	81.89	82.35	81.94	80.74	81.11	80.96
Specificity	99.00	99.19	99.10	99.18	99.42	99.47	99.46
Jaccard Index	83.67	76.61	76.48	76.56	76.90	77.62	77.39
Dice Index	91.11	86.75	86.67	86.73	86.94	87.40	87.25
Rover	0.9079	0.8189	0.8235	0.8194	0.8074	0.8111	0.8096
Rnonover	0.1773	0.2501	0.2533	0.2508	0.2426	0.2339	0.2366

^aZ = p00 + p10 × x + p01 × y+ ⋯ + p14 × xy⁴ + p05 × y⁵

A comparison between results obtained for the outer border with alternative interpolation and surface fitting methods is also included in Table III. As expected, performance of the existing interpolation and surface fit methods was mediocre and substantially underestimated vessel area.

D. Inter-Observer Variability

In affirmation of multiple previous studies showing excellent inter-observer variability in OCT, variability between expert annotations in this study was low. Error and Bland-Altman plots compare the two experts to each other (Fig. 9). The average absolute difference between the areas of their annotations was just 0.13±0.11 mm² (2.5±2.3%) and 0.88±0.75 mm² (9.7±8.6%) for the inner and outer border tracings, respectively; coefficients of determination between the two were greater than 0.99 and 0.93, respectively. Additionally, the average absolute radial distance between their inner and outer border annotations was just 0.07±0.07 mm and 0.21±0.23 mm, respectively. The distribution of radial distance between inner border tracings is centered around 0.02 mm with a standard deviation of 0.09 mm, while the distance between outer border tracings is nearly perfectly centered at 0.10 mm, and the standard deviation is 0.30 mm (Supplemental Fig. 4), further confirming the observation of strong agreement.

Fig. 9.

Comparison of two expert annotations of the inner and outer borders. (a) Error plot showing exceptional correlation between the two areas ($R_{inner}^2$ > 0.99, $R_{outer}^2$ = 0.93; dashed line shows 1-to-1 ratio); (b) Bland-Altman plot (average: 0.10 & 0.53 mm², standard deviation: 0.14 & 1.02 mm², respectively) illustrates no substantial systematic error (error is distributed).

E. OCT-IVUS Comparison

Agreement between cross-sectional areas delineated in corresponding OCT and IVUS frames show strong but imperfect agreement, as expected. Error and Bland-Altman plots compare areas delineated by an expert in IVUS to those delineated by both experts and the presented algorithm in OCT (Fig. 10). The average absolute difference between the areas of expert annotations for the inner and outer border tracings was 1.48±1.26 mm² (26.9±21.3%) and 1.49±1.39 mm² (16.4±13.4%), respectively. Comparing the algorithm results in OCT to the expert annotations in IVUS, the average absolute difference between the inner and outer border tracing areas was 1.41±1.17 mm² (25.0±20.1%) and 1.72±1.43 mm² (19.2±15.0%), respectively.

Fig. 10.

Comparison of the inner and outer borders delineated in OCT and IVUS. (a) Error plot showing correlation between the two areas delineated by human experts in OCT and IVUS ($R_{inner}^2$ = 0.66, $R_{outer}^2$ = 0.75; dashed line shows 1-to-1 ratio); (b) Error plot showing correlation between the areas delineated by the algorithm in OCT and a human expert in IVUS ($R_{inner}^2$ = 0.68, $R_{outer}^2$ = 0.67; dashed line shows 1-to-1 ratio); (c) Bland-Altman plot (average: 0.05 & 0.78 mm², standard deviation: 1.94 & 1.88 mm², respectively) illustrates minor systematic offset and greater absolute difference in larger vessels; (d) Bland-Altman plot (average: 0.25 & 0.72 mm², standard deviation: 1.82 & 2.12 mm², respectively) illustrates minor systematic offset and greater absolute difference in larger vessels.

IV. Discussion

A surface fitting approach inspired by Newtonian physics was highly effective in fitting variably-sparse and questionably-reliable data. Intuitively applying an anisotropic linear elastic mesh with a force function helped us autonomously delineate the outer border of a diseased vessel in OCT images. This capability has exciting implications, as it allows for the construction of 3D models of patient-specific vessel geometry from high-resolution OCT data for research and future clinical applications. Such models have been used to quantify and/or visualize 3D vessel properties, simulate interventions and their consequences through finite element and computational fluid dynamics analyses, and identify some cases of vulnerable plaques [30]–[33]. However, previous coronary artery anatomic reconstructions have been subject to limitations in application and/or fidelity arising from methodological approaches. Many modeling approaches have utilized either idealized geometries or relatively low-resolution image sources, such as IVUS, angiography, magnetic resonance imaging, or computed tomography, resulting in necessarily simplified models. Others have relied on histology of excised tissue, which precludes predictive studies, serial monitoring, or application to current patients. Modeling approaches to date utilizing high-resolution OCT have been limited by assumptions made during the reconstruction of the arterial wall; some use only the inner (lumen) border obtained from OCT (e.g. for computational fluid dynamics without fluid-structure coupling) [32], others have extrapolated radially from the inner border by a fixed distance to approximate the outer border [12], [15], while some have lofted circumferential cross-sections of indirectly-calculated diameters, defined only in plaque-free regions, to create external arterial wall surfaces [33]. The spring contour detection approach now proposed allows for the construction of models from in vivo geometry, without the limitations of idealization or unrealistic geometric assumptions, and with the benefit of the high resolution of OCT. This method can also be used in conjunction with existing reconstruction techniques that presuppose knowledge of the outer border location [34].

The method presented here is flexible, and may be applied to various other applications within and beyond image processing. In situations where gaps exist in the interior of data, where no or limited information is available, data points are noisy, or data are unevenly-distributed, the surface fitting technique can be used where others may struggle, providing estimates for the missing information and smoothed assessments of the known data. Moreover, as shown in the smoothing of the lumen, which was nominally detected with a previously validated method [12], the presented method can also be used to smooth complete sets of noisy, yet reliable, data. Furthermore, while optimization using curated datasets improves fit performance, parameters may be selected intuitively or heuristically to achieve reasonable fits. This feature sets the approach apart from others, such as polynomial fits, which provide drastically-varying results with slight modifications of input parameters. Finally, the intriguing ability to specify a force function provides a unique opportunity to incorporate complex features of all available data to improve the fit and resulting evaluation of underlying information. This is particularly useful in image analysis, where features may not be readily detected but hints about the likely position of such features may be extracted from the rich set of information embedded within the image. Thereby, insights from other studies, including machine learning, pattern recognition, and other analyses, may be incorporated.

Numerous opportunities exist to enhance and improve the performance of the fitting technique in this application. Accuracy of the current approach is primarily limited by two factors: inner and candidate outer border segment detection. Error in lumen border detection generally arises in the presence of image acquisition artifact, such as insufficient blood flush or irregular catheter rotation, as well as side-branches. Lumen detection is important to the outer border detection process, as it is used to flatten the image prior to detection of the contours that are likely segments of the outer border—errors in lumen detection thus skew or distort the flattened image. However, because the outer surface fitting is performed in the flattened state (to avoid arbitrary penalization of contours arising due to catheter position within the lumen), any error in detecting the lumen is directly introduced to the outer border when the fit surface is reconstituted to the non-flattened state. Therefore, improved techniques to detect the lumen, particularly those robust to artifact presence [35], [36], will have a direct positive impact on the performance of this technique.

Detection of visible segments of the outer border is also a process step primed for improvement. While segments were identified by the presented approach in just over half of all radial projections, some segments visible to the trained eye were not identified. Of greater concern are the candidate segments identified that did not correspond to the actual outer border. These segments usually arose at the adventitiaperiadventitia transition or, more problematically, within heterogeneous plaques where the selected grayscale pattern occurred by happenstance (e.g. when thin bands of calcified tissue were present). In this latter case of false segment detection, the existence of the falsely-identified outer border segments skewed the surface in a way that led to underestimation and misrepresentation of some regions with high plaque burden. However, the fitting approach tempered these negative effects, especially when the false detected segments were small relative to the correctly-identified segments (as they frequently were). Nevertheless, improved approaches will be useful. One possible alternative detection method of note has been recently reported for segmentation of healthy wall regions [17], and other methods similarly derived from earlier segmentation techniques developed for retinal OCT imaging [37] are sure to follow. Additional strategies for segment detection improvement could include explicit incorporation of penetration depth, the external limit of which is not currently considered by the algorithm. Inclusion of this factor could be used to delimit a region of interest within a proximal subset of the image, thereby eliminating opportunities for false detection beyond this region and reducing computational operations in regions of the image in which no viable information is likely to be present.

Finally, an additional opportunity for improving performance is implementation of a more sophisticated force function (F_ij, (17)) and/or base layer (L_ij). Such functions could leverage additional a priori knowledge and insights gained through previous studies of typical plaque and vessel wall morphology (akin to the approach taken by Kok et al. to develop a plaque-specific generalized estimation equation model of necrotic core thickness [34]), perhaps incorporating plaque tissue characterization studies [12]. Future work will explore these possible avenues of enhancement.

There are certain limits to the applicability of the technique, as well as the results of this study. As it stands now, this surface fitting technique does not allow for folds in the surface, which would be represented as non-singularities in the contour map with multiple nodes in a single column. Similarly, the surface must intersect each column, such that intentional gaps cannot be achieved. Modification of the presented approach using the same underlying physical concepts could allow for application in either scenario. Another limitation pertains to parameter selection and generalizability. While the dataset included 7 distinct image datasets (pullbacks), convergence testing using information from increasing numbers of randomly-selected image frames (Fig. 4) showed that optimal parameters—and subsequent overall performance—did vary significantly depending on the number and subset of frames used. Yet, the strong convergent trend, which is generally stabilized when at least 30 (of the 724) frames were used, suggests that the final parameters should be close to optimal for all OCT datasets for this application. Parameters for other applications will surely differ, and will likely require training datasets of their own to determine. Furthermore, optimization for other criteria may be preferable for this and other applications. For example, rather than minimizing summed radial distance between the surface and the expert annotation of the training set, it could be argued that smoothness or an overlap comparison metric (e.g. sensitivity, specificity, R_over, R_nonover, etc.) are more important and relevant metrics. Such changes would result in different minimization functions ((16) and (18)), and subsequently different fit parameters (K₂/K₁, K₃/K₁, K₄/K₁, and α/K₁).

Finally, regarding the validation of this method, expert annotations in OCT were used as the primary reference, but are not considered the clinical gold standard for identifying the outer border of the vessel. Due to the limited penetration depth of OCT, IVUS is the clinical gold standard for measurements involving the outer border, as the border is almost universally visible, regardless of plaque burden [5]. In contrast, and as noted earlier, the outer border is often not visible in OCT within regions with substantial lipid plaque. However, evidence in literature indicates that outer border estimations made in OCT by expert interventional cardiologists are generally quite accurate compared to annotations made with clinical gold standard IVUS [38]. Because only annotations made within the same OCT images could be used in directly validating the automatic detection (i.e. through annotation distance and overlap metrics), these were considered the primary validation reference for this purpose. Nevertheless, annotated cross-sectional areas in co-registered IVUS images were included for comparison to incorporate the clinical gold standard in validation and to test for viability in clinical scenarios. As expected, agreement between the algorithm results and IVUS expert annotations was not as good as agreement with OCT expert annotations, though this largely reflected the difference between IVUS and OCT expert annotations (Fig. 10). Differences between measurements made in the two modalities have been reported elsewhere, even in well-controlled phantom models [39], and compounding errors arising from physiological state disparities between non-concurrent acquisitions and imperfect co-registration amplify these differences. To avoid these magnifying errors, other work in the field has leveraged phantom models of known, directly measureable dimensions for validation purposes [13], [39]. While exceedingly useful for validating imaging technology and evaluating measurements of inner surfaces and their contents (e.g. stents), measurement of the outer border does not offer a useful validation mechanism due to the lack of physiologically-relevant optic properties and feature patterns.

V. Conclusion

In an exciting advancement, a smooth surface fitting method implementing an anisotropic linear-elastic mesh and force function overcomes challenges and shortcomings of existing interpolation and surface fitting techniques. Use of equilibrated mechanical lumped elements helps reconstruct and smooth surfaces in the face of partial, imperfect, or incomplete information. With this efficient, flexible, and intuitive approach, we have successfully demonstrated, for the first time, the capability to autonomously delineate the entire outer border of diseased vessels in OCT images. The method is even more intriguing given that extension of this work could assign values and attributes to physical parameters that are extracted from the architecture and morphology of a blood vessel wall; fit parameters derived from the data might well offer deeper meaning and allow inferences about vessel state and function.