Multiquadric Spline-Based Interactive Segmentation of Vascular Networks

Abstract

Commonly used drawing tools for interactive image segmentation and labeling include active contours or boundaries, scribbles, rectangles and other shapes. Thin vessel shapes in images of vascular networks are difficult to segment using automatic or interactive methods. This paper introduces the novel use of a sparse set of user-defined seed points (supervised labels) for precisely, quickly and robustly segmenting complex biomedical images. A multiquadric spline-based binary classifier is proposed as a unique approach for interactive segmentation using as features color values and the location of seed points. Epifluorescence imagery of the dura mater microvasculature are difficult to segment for quantitative applications due to challenging tissue preparation, imaging conditions, and thin, faint structures. Experimental results based on twenty epifluorescence images is used to illustrate the benefits of using a set of seed points to obtain fast and accurate interactive segmentation compared to four interactive and automatic segmentation approaches.

I. Introduction

Over the past two decades interactive methods for clinical and biomedical image segmentation have been investigated since the pioneering work of Live-Wire, Live-Lane [1] and Intelligent Scissors [2]. Fully automatic image segmentation is essential for quantitative analysis but remains an unsolved problem, so user driven interactive methods continue to be a powerful alternative when extremely precise segmentation is required. However, manual methods although routinely used are tedious, time-consuming, expensive, inconsistent between experts and error prone. In semi-supervised interactive segmentation the goal is for the user to provide a small amount of partial information or hints for an automatic algorithm to use in order to produce accurate boundaries suitable for the user. The coupled interaction between the user provided input and the semi-supervised segmentation algorithm should be minimal and robust.

Commonly used drawing tools for interactive segmentation interfaces include active contour or boundary drawing, scribbles to identify foreground and background regions, and rectangles to outline the object of interest. In the case of complex anatomy, tissue structures and imaging conditions, even such interactive segmentation tools would still require a significant amount of user intervention to achieve accurate segmentation results. An example of a difficult case is the vessel network segmentation for the epifluorescence image of dura mater microvasculature shown in Figure 1. Since the vessel structures are thin, manually drawing contours, scribbles or rectangles using current interactive segmentation tools, is tedious, slow and often inaccurate, creating a need for better semi-automated delineation tools [3], [4]; the latter includes survey of software tools for reconstructing curvilinear structures. Our motivation is to develop quantitative tools to study the effects of hormone therapy on angiogenesis and vascular remodeling of capillaries [3], [4].

Fig. 1.

The proposed interactive MQ segmentation result using 29 seeds produces accurate results. (a) Dura mater vessel image 012706-ERBKO-05 (contrast enhanced for visualization), (b) ground-truth (GT) vessel segmentations provided by an expert physiologist. (c) result of MQ segmentation without pre- or post-processing (Dice 0.8872).

In this paper we propose a new paradigm of using only labeled seed points, that do not need to carefully follow centerlines or boundaries for user assisted segmentation of difficult biomedical imagery with curvilinear structures. Seed points are much easier and more intuitive for the user to provide labeled inputs and requires minimal interaction compared to contours, lines, rectangles or other shapes. We introduced point-based segmentation using elastic body splines and Gaussian elastic body splines in [5]–[7] and showed their advantages for segmenting natural imagery using an average of less than ten seed points. In this paper, we focus on characterizing the utility of using interactive seed points for segmenting thin vessels in epifluorescence imagery and extend the basis set to multiquadric splines. Figure 1 shows a case of using 9 foreground and 20 background seed points to produce a highly accurate segmentation result with a Dice value of 88.7 percent based on scoring the overlap with the manual ground-truth.

Many techniques exist for automatic segmentation of vessels, especially in angiogram imaging for characterizing neurological, retinal, and heart regions [8]. The majority of these are fully automatic segmentation methods which do not require any user intervention. Interactive segmentation methods specialized for thin curvilinear structures provide flexibility to the user in defining and refining object boundaries. Interactive segmentation based on techniques such as Graph-Cut [9] and Random Walker [10] are popular approaches where the user provide scribbles on the object of interest and the background. In Live-Wire [1] and Intelligent Scissors [2] the user carefully draws along the object boundary edges in a semi-automated manner. Being edge-based methods they perform poorly in the presence of weak edges and noise. Recently in [5] and [6] we proposed using sparse set of seed points to perform a user driven image segmentation. Sparse inputs leads to a very fast image segmentation process.

In this work we utilize multiquadric (MQ) splines for point-based interactive segmentation of thin curvilinear structures in epifluorescence imagery. Unlike other semiautomated tools [4] our interactive segmentation framework requires only a few labeled seed points and can obtain vessel segmentations under inhomogeneous background, partially diffused vessels and uneven contrast. We frame the interactive image segmentation task as a semi-supervised interpolation problem which uses MQ as the basis function in the governing interpolation equation. This work is part of a system that builds upon our previous work on multi-focus fusion and edge preserving smoothing [11]–[13].

The rest of the paper is organized as follows. Section II introduces our multiquadric splines-based interactive image segmentation framework. Section III provides detailed experimental results on epifluorescence imagery of mice dura mater capillaries and quantitative performance comparison with other interactive and global thresholding methods followed by conclusions.

II. Multiquadric Splines for Interactive Image Segmentation

A. Interactive segmentation with splines

Epifluorescence imagery of tissues have high variability in both arteriole and venule vessel boundaries. We consider semi-automated segmentation framework based on user defined seed points to obtain reliable segmentations. For this purpose we learn the parameters for a binary classifier function d_f (x⃗) of the following form from the user supplied inputs,

$$d_f(\overrightarrow{x})=\sum _{i=1}^m{a}_i{x}_i+b+\sum _{j=0}^{N}g(\overrightarrow{x}-\overrightarrow{x_j})c_j.$$ (1)

such that,

$$\sum _{j=0}^N{c}_j{x}_{j,k}=\overrightarrow{0}k=1,\dots ,5$$ (2)

where x⃗ = [x₁, x₂, . . . , x₅]^T and x_i is the i^th feature, m is the number of features (m = 5 in our case), c_j is the spline coefficient and a_i and b are coefficients of the linear term, j is the feature index, N is the total of foreground and background seed points and g is the multiquadric spline function given by,

$$g(\overrightarrow{x}-{\overrightarrow{x}}_j)=\sqrt{d^2(\overrightarrow{x},{\overrightarrow{x}}_j)+R^2},$$ (3)

where d is the distance between the data points x⃗ and x⃗_j. In our case d² = ||x⃗_i–x⃗_j||² and R² is a user defined positive constant. This function is similar to the extensor function for interpolation [14]. Equations in (1) constitute a linear system of equations and the constraint in (2) ensures that the system of equations has a unique solution. In equation (1) the first term is the linear part of the interpolation function while the second term represents the non-linear part of the interpolation function. For the task of interactive segmentation d_f (x⃗) is learned from the user labeled seed points such that d_f (x⃗)) = +1 for the foreground pixels and d_f (x⃗) = −1 for the background pixels.

Once the MQ interpolation spline function (1) is learned, we can threshold the interpolation function d_f (x⃗) at zero and assign label ℓ(x⃗),

$$\ell (\overrightarrow{x})=\{\begin{array}{ll}\mathit{foreground},\hfill & \text{if}{d}_f(\overrightarrow{x})\ge 0\hfill \\ \mathit{background},\hfill & \text{if}{d}_f(\overrightarrow{x})<0.\hfill \end{array}$$ (4)

For the interactive image segmentation task we use five image features at each pixel, namely three color features (Red, Green, Blue) converted to the LAB color space and two coordinate locations of each seed pixel. For the interpolation task the use of coordinate location as a feature is critical as it effects the influence of seed points and provides geometric spatial regularization.

B. Multiquadric splines

In this work we use multiquadric (MQ) splines as they provide robust interpolation of data points in high dimensional spaces. Let w⃗ be the vector of all the MQ coefficients given as,

$$\overrightarrow{w}=\left[\begin{array}{llll}{\overrightarrow{c}}_F^T\hfill & {\overrightarrow{c}}_B^T\hfill & {\overrightarrow{a}}^T\hfill & b\hfill \end{array}\right]$$ (5)

where c⃗_F and c⃗_B are coefficients corresponding to the N_f, foreground and N_b background seed pixels.

$${\overrightarrow{c}}_F={\left[c_1\dots c_{N_f}\right]}^T,{\overrightarrow{c}}_B={\left[c_1\dots c_{N_b}\right]}^T,\overrightarrow{a}={\left[a_1\dots a_5\right]}^T$$

The MQ coefficients are estimated by solving the matrix equation,

$$\overrightarrow{w}={\mathbf{L}}^{-1}\overrightarrow{Y}$$ (6)

where,

$$\mathbf{L}=\left[\begin{array}{cc}\mathbf{K}& \mathbf{P}\\ {\mathbf{P}}^T& \mathbf{O}\end{array}\right],\mathbf{K}=\left[\begin{array}{cc}{\mathbf{G}}_{FF}& {\mathbf{G}}_{FB}\\ {\mathbf{G}}_{BF}& {\mathbf{G}}_{BB}\end{array}\right]$$ (7)

$${\mathbf{G}}_{FF}(\overrightarrow{r})=\left[\begin{array}{ccc}{\mathbf{G}}_{11}(r_{11})& \dots & {\mathbf{G}}_{1f}(r_{1f})\\ ⋮& & ⋮\\ {\mathbf{G}}_{N_{f}1}(r_{11})& \dots & {\mathbf{G}}_{N_{f}f}(r_{N_{f}f})\end{array}\right]$$ (8)

with r_ij = d(x⃗_i − x⃗_j) where d is the distance between the feature vectors x⃗_i and x⃗_j, d² = ||x⃗_i−x⃗_j||². G_FF is the matrix of MQ (3) spline functions defined only over the foreground pixels.

$$G(i,j)=g({\overrightarrow{x}}_i-{\overrightarrow{x}}_j)=\sqrt{d^2({\overrightarrow{x}}_i,{\overrightarrow{x}}_j)+R^2}$$

Similarly G_FB, G_BB and G_BF are also defined. We have,

$$\mathbf{P}=\left[\begin{array}{cc}{\mathbf{P}}_F& {\mathbf{I}}_F\\ {\mathbf{P}}_B& {\mathbf{I}}_B\end{array}\right],$$ (9)

where the identity matrices are given by,

$${\mathbf{I}}_F=[{\mathbf{I}}_1\dots {\mathbf{I}}_{N_f}],{\mathbf{I}}_B=[{\mathbf{I}}_1\dots {\mathbf{I}}_{N_b}],$$ (10)

with I is 5 × 5 identity matrix, and

$${\mathbf{P}}_F=\left[\begin{array}{ccc}{x}_{11}& \dots & x_{15}\\ ⋮& & ⋮\\ x_{N_{f}1}& \dots & x_{N_{f}5}\end{array}\right]$$ (11)

where x_ij is the i^th foreground pixel for j^th feature. I⃗_F and I⃗_B are vectors of 1. P_B is similarly defined. The vector Y⃗,

$$\overrightarrow{Y}=\left[\begin{array}{ccc}{\overrightarrow{Y_F}}^T& {\overrightarrow{Y_B}}^T& {\overrightarrow{O}}^T\end{array}\right]$$ (12)

$$\overrightarrow{Y_F}={[1\dots 1]}^T,{\overrightarrow{Y}}_B={[-1\cdots -1]}^T,\overrightarrow{O}={[0\dots 0]}^T$$

Y⃗_F = [1 . . . 1]^T, Y⃗_B = [−1 . . . −1]^T, O⃗ = [0 . . . 0]^T consists of Y⃗_F with values +1 for the foreground seed points and Y⃗_B with values −1 for the background seed points, O⃗ is a vector of zeros. We refer to [5] and [6] for more details for solving the above matrix problem (6).

III. Experimental Results

The experiments were performed using high resolution epifluorescence images of mice dura mater acquired using a video microscopy system (Laborlux 8 microscope from Leitz Wetzlar, Germany) equipped with 75 watt xenon lamp and QICAM high performance digital CCD camera (Quantitative Imaging Corporation, Burnaby, Canada) with 0.56 micron per pixel resolution. Our dataset consists of 10 wild-type (WT), and 10 knock-out (KO) mice dura mater epifluorescence images stained using Alexa Fluor 488-conjugated soybean agglutinin (SBA lectin). We collected manually drawn ground-truth (GT) with supervision from an expert physiologist for benchmarking the segmentation methods. Note that the ground truth for these images reported in the Table I has since been updated. The overall biological motivation is to quantify the role of estrogen receptors (ERβ) in microvasculature remodeling in WT and KO mice.

Table I.

Comparison of different segmentation methods on 20 (10-knock-out (012706-ERbKO) + 10-wild-type (012606-ERbWT)) mice dura mater epifluorescence images. We show the Dice values for our proposed multiquadric spline interactive segmentation method, compared with automatic thresholding as well different interactive segmentation approaches. Higher Dice values indicate better results compared to manual gold-standard ground-truth segmentations.

No.	Type	Niblack [11]	Otsu	Random Walker [10]	Graph Cut [9]	MQ
1	KO-01	0.6324	0.3038	0.2177	0.7301	0.7092
2	KO-03	0.5456	0.6813	0.3404	0.8528	0.8031
3	KO-04	0.6196	0.6282	0.3176	0.8023	0.8436
4	KO-05	0.7970	0.5780	0.5145	0.7097	0.8367
5	KO-06	0.7452	0.5999	0.6331	0.8080	0.8641
6	KO-08	0.6875	0.6406	0.5322	0.8154	0.8557
7	KO-09	0.4363	0.4907	0.2085	0.6018	0.7972
8	KO-10	0.5736	0.6714	0.5293	0.7259	0.7497
9	KO-15	0.7909	0.6238	0.5721	0.7660	0.8587
10	KO-19	0.7882	0.6640	0.6388	0.7128	0.7261
11	WT-01	0.6665	0.4903	0.6063	0.7582	0.7647
12	WT-03	0.4095	0.4410	0.3824	0.6766	0.7375
13	WT-04	0.5726	0.4974	0.3623	0.6818	0.7381
14	WT-05	0.7589	0.6099	0.5844	0.8126	0.8637
15	WT-06	0.8467	0.6673	0.6343	0.8520	0.8466
16	WT-07	0.8296	0.6348	0.7835	0.9448	0.9224
17	WT-08	0.7436	0.6108	0.7746	0.8441	0.8555
18	WT-09	0.7116	0.5457	0.4694	0.7010	0.7869
19	WT-12	0.6774	0.5689	0.8506	0.7540	0.8411
20	WT-14	0.8772	0.6520	0.6915	0.8094	0.7979
Average		0.6855	0.5800	0.5328	0.7680	0.8099

Figure 2(a,f,k) shows three example epifluorescence images of mice dura mater (2 WT, 1 KO) vasculature networks used in the experiment along with a sparse set of user provided seed points sampling the foreground (yellow) and background (red). Figure 2(b,g,l) shows the gold standard GT binary image (foreground vessels in white and background in black). We show a comparison of our proposed multiquadric (MQ) spline-based semi-supervised interpolation (Figure 2(c,h,m)) with interactive segmentation methods based on Random Walker (RW) [10] (Figure 2(d,i,n)) and Graph-Cut (GC) [9] (Figure 2(e,j,o)). As can be seen, our method obtains good segmentations overall. The Random Walker interactive segmentation method fails to capture branching vessels and obtains spurious foreground regions. The Graph-Cut based approach fails to segment diffuse vessel boundaries, in particular venules where the vessels have very low contrast. Our approach obtained good segmentation results while requiring very few seed points for foreground vessels. The last column shows an image where all three methods produce low Dice scores, though our approach still achieves better segmentation.

Fig. 2.

Quantitative comparison of semi-automated interactive segmentation results shown for three sample microvasculature images. From left to right: Input images (012606-ERbWT-07, 012706-ERbKO-06, 012606-ERbWT-04) with user selected foreground (fg) and background (bg) seeds, ground truth mask, Multiquadric (proposed method) MQ, Random walk [10] RW, Graph-cut [9] GC. No pre- or post-processing was performed to generate these segmentation results. Our proposed MQ method obtains better segmentations without spurious foreground segments and the Dice values indicate we obtain closer results to GT.

To evaluate the performance of different segmentation methods quantitatively, we use the Dice similarity coefficient,

$$\mathit{Dice}(P,Q)=\frac{2\mid P\cap Q\mid }{\mid P\mid +\mid Q\mid },$$

where P and Q are the representations of automatic and ground-truth (GT) segmentations. Values closer to one indicate better performance in terms of the physiologist expert verified gold standard. Table I shows the dice values for 10 wild-type and 10 knock-out mice dura mater epifluorescence images. Our method in general outperforms related interactive segmentation methods as well some well-known automatic thresholding methods [11] from the literature. Note that we applied no pre- or post-processing methods to generate these segmentation results.

IV. Conclusions

In this paper, we considered using a seed point-based interactive image segmentation method for microvasculature network extraction from epifluorescence imagery. Using a binary classification function based on multiquadric splines our method was able to obtain reliable segmentations using only sparse seed points from the user. Seed points are much easier and more intuitive for the user to provide. In this case for vessel segmentation seed points are clearly superior to other interactive methods avoiding the need to trace difficult to discern boundaries, drawing very thin scribbles or outlining many bounding boxes. Experimental results on a set of mice dura mater epifluorescence images indicate our results are better than other fully automatic or interactive segmentation methods. We are currently integrating our semi-automated interactive vessel segmentation method for more rapid ground truth generation and within an automatic image analytics framework for understanding the influence of physiological processes on microvasculature morphology.