Retinal vessel segmentation on SLO image

Abstract

A scanning laser ophthalmoscopy (SLO) image, taken from optical coherence tomography (OCT), usually has lower global/local contrast and more noise compared to the traditional retinal photograph, which makes the vessel segmentation challenging work. A hybrid algorithm is proposed to efficiently solve these problems by fusing several designed methods, taking the advantages of each method and reducing the error measurements. The algorithm has several steps consisting of image preprocessing, thresholding probe and weighted fusing. Four different methods are first designed to transform the SLO image into feature response images by taking different combinations of matched filter, contrast enhancement and mathematical morphology operators. A thresholding probe algorithm is then applied on those response images to obtain four vessel maps. Weighted majority opinion is used to fuse these vessel maps and generate a final vessel map. The experimental results showed that the proposed hybrid algorithm could successfully segment the blood vessels on SLO images, by detecting the major and small vessels and suppressing the noises. The algorithm showed substantial potential in various clinical applications. The use of this method can be also extended to medical image registration based on blood vessel location.

I. INTRODUCTION

The appearance of a blood vessel, i.e., diameter, color and tortuosity is an important indicator, often used for detection of eye diseases, such as diabetic retinopathy, a leading cause of blindness worldwide. The blood vessels can also be used as a landmark to register the retinal images of the same patient taken at different visits, or even taken with different ophthalmic devices.

Spectral domain optical coherence tomography (SD-OCT), a new generation of ophthalmic imaging device, detects the light back-scattered from various layers of retina to generate a two-dimensional, cross-sectional image of ocular tissue and is capable of obtaining three dimensional (3D) images of the scanned region. Although the actual scanning time is less than 2 seconds, subtle eye motions during the scanning cause enough distortion in the resulting images. This is clinically problematic since longitudinal observation of the same location is essential. Scanning laser ophthalmoscopy (SLO) images taken from an SD-OCT device can be used as a reference for image registration in order to improve visualization. The SLO image is typically a grey scale image with non-uniform illumination, low global/local contrast and high background noise, which makes vessel segmentation on SLO images a challenge.

Niemeijer et al. [1] gave a comparative study of retinal vessel segmentation algorithms on conventional fundus photographs (retinal images). Existing vessel segmentation techniques on the conventional fundus photographs can be divided into several major categories: matched filter based method [2, 3], thresholding based method [4], tracking based method [5], mathematical morphology method [6], and classification based method [7, 8]. The matched filter method is convolving the image with a directional filter designed according to the vessel profile [3], such as Gaussian [2] and second-order Gaussian filter [3]. The matched filter can be used as the first step for other segmentation methods. The filter response enhances the vessel pattern features, thus improving the performance of thresholding or tracking process. Hoover and et al. [4] introduced a piecewise thresholding probe algorithm on the matched filter response (MFR) image, yielding a 90% true positive detection rate compared with the manual label ground truth. Staal and et al. [7] described a pixel classification method for color retinal image. Feature vectors were first computed for each pixel and then a kNN-classifier was used to classify the given pixel as either vessel or non-vessel. It is a supervised learning method, which requires manually labeled images for training. Color images provide much more information, which is not available in SLO images. The algorithms in previous studies did not work perfectly on SLO image due to the low global/local contrast, non-uniform illumination and noise.

In this paper, a hybrid algorithm is proposed to automatically segment blood vessels on SLO images, which includes several steps. The original SLO image is first transformed into a feature response image using four methods, which are designed based on different combinations of matched filter, contrast enhancement and mathematical morphology operators. Threshold probing algorithm [4] is then applied to all the four response images. The final vessel map is generated by taking the weighted majority opinion from the four vessel segmentation results pixel by pixel, which combines both the original and enhanced image information to solve the above mentioned problems.

II. METHODOLOGY

The proposed algorithm combines several operations, i.e., matched filter, contrast enhancement, mathematical morphology and thresholding probe. The overall hybrid method could be divided into three steps, (1) preprocess the SLO images to generate feature response images, (2) apply thresholding probe algorithm on those response images to segment blood vessels, and (3) generate final vessel map.

Blood vessels on SLO image are difficult to detect by only the single global threshold. Matched filter or morphology operations can efficiently enhance the vessel pattern features in order to improve the performance of thresholding method. Adaptive thresholding performs well on the images having non-uniform illumination, though the low contrast still poses a substantial challenge to the algorithm performance. Contrast enhancement can enhance the vessels in the low contrast region, but introduces more noises simultaneously. A single process may not solve all the problems of vessel segmentation. The proposed hybrid algorithm takes the advantages of multiple approaches, i.e. enhancing weak vessels and keeping the original information from low noise image, so that the final vessel map is more stable than each single method.

The methodology section is organized as follows. Sections A to D introduce four different operators in the hybrid algorithm; the overall hybrid algorithm is described in section E.

A. Matched filter

Blood vessels usually appear darker relative to the background in SLO images due to lower reflectance compared to the retinal surface [2]. They have small curvatures that may be approximated by piecewise linear segments [2]. Gang and et al. [3] indicated that the background contributed less to the convolution response using second-order Gaussian model compared to Gaussian model [2]. Therefore a normalized two dimensional (2D) second-order Gaussian filter template (Fig. 1(a)) is used to approximately represent the pattern along the direction of vessel length, written as (modified based on eq.(10) in [3]):

$$\begin{array}{cc}\hfill G(x,y)& =\mathit{\text{kR}}(\theta )h(x,y)=\mathit{\text{kR}}(\theta )\frac{1}{\sqrt{2\pi {\sigma }^2}}(x^2-{\sigma }^2)e^{-(\frac{x^2}{2{\sigma }^2})}\hfill \\ \hfill y& \le L,\left|x-x_0\right|\le W/2,k={\mathrm{\Sigma }}_{x,y}R(\theta )h(x,y)\hfill \end{array}$$ (1)

where L (the vessel direction) and W are the length and width of filter template respectively, x₀ is the center location along the vessel direction, k is the normalized coefficient, and R is the rotation matrix according to vessel orientation. Directional filters are generated using the rotation matrix R with the rotation angle θ. Multiple σ corresponding to different vessel widths are also applied to generate different filters. All the filters are employed on the SLO image and the response image takes the maximal value pixel by pixel among those responses according to different σ and θ.

Fig. 1.

2D matched filter and mask

B. Local contrast enhancement

Local enhancement is introduced as follows:

$$f_{\mathit{\text{enhance}}}(x,y)=255\frac{f(x,y)-\text{min}(f(x+i,y+j))}{\text{max}(f(x+i,y+j))-\text{min}(f(x+i,y+j))}i,j\le W/2$$ (2)

where f(x,y) is the original image and W is the window size. The intensity values are normalized to the range of 0 to 255 within the given window. If the window size is enlarged to the image size, it becomes a global enhancement. Weak vessels could be enhanced by this operation with consequent increase in the noise level.

C. Mathematical morphology

According to Zana and Klein’s study [6], a line structure was selected to generate a morphological filter for vessel segmentation assuming the vessels maintain predominantly local linear orientation. Basic morphological operators are erosion, dilation, opening, closing as defined in [6]. The top-hat, bottom-hat, and morphological enhancement operators are written as

$$\begin{array}{c}{f}_{\mathit{\text{top}}-\mathit{\text{hat}}}(x,y)=f(x,y)-f_{\mathit{\text{opening}}}(x,y)\hfill \\ f_{\mathit{\text{bottom}}-\mathit{\text{hat}}}(x,y)=f_{\mathit{\text{closing}}}(x,y)-f(x,y)\hfill \\ f_{\mathit{\text{enhance}}}(x,y)=f_{\mathit{\text{top}}-\mathit{\text{hat}}}(x,y)+f(x,y)-f_{\mathit{\text{bottom}}-\mathit{\text{hat}}}(x,y)\hfill \end{array}$$ (3)

Because blood vessels are darker than the background in SLO image, a bottom-hat operator and enhancement operator are used to generate the feature response images. Morphological filters rotated to different directions are applied on the image. The final filter response is the maximum (measuring pixel by pixel) among those responses of directionally morphological filters with different rotation angles.

D. Thresholding probe

Thresholding probe [4] is an adaptive thresholding with region growing algorithm for vessel segmentation. The method can be summarized into a number of steps: (1) Generating a MFR image by convolving the image with the matched filter, (2) detecting the endpoints of the vessel tree using single global threshold to initialize a probe queue, (3) starting from the points in the queue, probe growing with local threshold to generate a vessel tree, (4) re-computing the endpoints of vessel tree to update probe queue, and (5) repeating step 3 and 4 until the queue is empty. Optic disc and image borders are masked in this operation (Fig. 1(b)).

E. Hybrid algorithm

Hybrid algorithm includes thresholding probe operator and different preprocessing operators. Four vessel segmentation methods (written as M1 to M4) are designed by taking different combinations among four introduced operations, as shown in Fig. 2. By observation, SLO images usually have low global or local contrast. Therefore the resulted vessel map from method 4 (in which local contrast enhancement is operated on the original SLO image) is considered having higher contribution to generate the final result (written as M). Let 1 and 0 indicate vessel and non-vessel respectively. Majority opinion is taken pixel by pixel from the four resulted vessel maps to generate the final vessel map, described as

$$M(x,y)=\{M_1(x,y)+M_2(x,y)+M_3(x,y)+2M_4(x,y)\}>3$$ (4)

where method 4 has higher weight than all the other three methods. The M(x,y) is labeled as 1 if the weighted summation is larger than 3, otherwise it is labeled as 0. The hybrid algorithm combines different operations so as to enhance the vessel pattern and suppress the effect of noises.

Fig. 2.

Hybrid method of blood vessel segmentation

III. RESULT AND DISCUSSION

20 SLO images (668×801 pixels, taken from healthy eyes) acquired at University of Pittsburgh Medical Center Eye Center were tested by the proposed hybrid algorithm of blood vessel segmentation. All the images were resized to 334×400 pixels in order to reduce the computation time. Fig. 3 illustrates an example of the entire segmentation procedure. Feature response images before thresholding probe operation generated from four designed methods are shown in Fig. 3(c)–(f). The vessel maps after employing the thresholding probe are illustrated in Fig. 3(g)–(j). The final vessel map was obtained by taking the weighted majority opinion (Fig. 3(b)), in which the weak vessels were remained and the segmentation errors from each single method were removed.

Fig. 3.

An example of hybrid algorithm of retinal vessel segmentation. (a) SLO image taken from SD-OCT, (b) Final vessel map generated through weighted majority opinion, (c)–(f) Feature response images of method 1 to method 4 before applying thresholding probe operation, (g)–(i) Vessel maps of method 1 to method 4.

Different operations have their own advantages and disadvantages. Thresholding on the feature response image much improved the performance compared to applying it directly on the original image. Matched filter and mathematical morphology operations enhanced the vessel pattern features and suppressed the noises. However, it was difficult to obtain a high vessel response with both methods in regions with low contrast as shown in the right region of Fig. 3(g)(i). Global or local contrast enhancement operation solved the low contrast problem, but introduced more noise as illustrated in Fig. 3(h)(j). The proposed hybrid algorithm effectively took those advantages and reduced the probability of error measurements.

Taking the weighted majority opinion is a simple and fast method to fuse the vessel maps and generate a final result. Advanced methods, such as machine learning, could be further studied to fuse the vessel maps, which may have a more robust output.

The proposed hybrid algorithm was also applied on the OCT enface images (generated by averaging 3D OCT data along z axis) and obtained similar results. This indicated that the proposed algorithm is not limited in the SLO images, which can be expanded to multiple applications.

IV. CONCLUSION

A hybrid algorithm of retinal vessel segmentation has been proposed in this paper, using matched filter, mathematical morphology, contrast enhancement and thresholding probe. Weighted majority opinion was taken to generate the final segmentation result. Experiment results showed that the proposed algorithm is a more efficient way to segment the vessels, suppress the noises and remove error measurements compared to every individual method. The proposed hybrid algorithm showed a substantial potential in various clinical applications, such as qualification of blood vessel appearance and medical image registration based on blood vessel location.