Optic Disc and Cup Segmentation from Color Fundus Photograph Using Graph Cut with Priors

Abstract

For automatic segmentation of optic disc and cup from color fundus photograph, we describe a fairly general energy function that can naturally fit into a global optimization framework with graph cut. Distinguished from most previous work, our energy function includes priors on the shape & location of disc & cup, the rim thickness and the geometric interaction of “disc contains cup”. These priors together with the effective optimization of graph cut enable our algorithm to generate reliable and robust solutions. Our approach is able to outperform several state-of-the-art segmentation methods, as shown by a set of experimental comparisons with manual delineations and a series of results of correlations with the assessments of a merchant-provided software from Optical Coherence Tomography (OCT) regarding several cup and disc parameters.

1 Introduction

As the second leading cause of blindness in the United States, glaucoma is characterized by progressive optic nerve damage and defects in retinal sensitivity leading to loss of visual function. Early diagnosis and optimal treatment have been shown to minimize the risk of visual loss due to glaucoma. The hallmark of glaucomatous progression is optic disc cupping, a phenomenon of enlargement of cup which is a small crater-like depression seen at the front of the optic nerve head (ONH), as shown in Fig. 1. Cupping is caused by little nerve fibers wiped out along the rim of the optic nerve in glaucoma.

Fig. 1.

A glaucomatous patient with progressed optic nerve cupping as shown by the color fundus images of optic nerve head (ONH) taken at an early time (left picture) and a late time (middle picture) after about 53 months. The right picture shows the annotation of main ONH landmarks. Neuroretinal rim, defined as the region between optic disc and cup boundaries, manifest as a “lantern ring” around the optic disc center.

Ocular imaging instruments provide clinically useful quantitative assessment of optic disc cupping. However, fully automated ONH quantification techniques are still lacking and it is still a clinical standard to perform subjective optic disc assessment using disc photographs. Conventional ONH evaluations include subjective assessment by observing the pair of stereo fundus photographs and computerized planimetry which allows for generating several disc parameters based on manual labeling of disc and cup margins [1]. However, these conventional methods are time consuming, labor intensive and prone to high intra/inter-observer variabilities.

Automatic segmentation of optic disc and cup can help to eliminate the disadvantages of the conventional ONH evaluation methods; nevertheless, it is surprisingly difficult. There have been efforts to automatically segment optic disc and cup from color fundus photographs, which can be classified as techniques based on template matching [2], machine learning [3], active contour model [4], level sets [5, 6] and Hough transform [7]. Our experiments showed that most of these existing techniques are incapable of generating accuracy and robustness enough for evaluating ONH in clinical practice. The deteriorations of these algorithms first come from the challenges of the involved task, caused by the blurry and faint boundaries of optic disc and cup, large inter-subject variability in ONH appearance, interference of blood vessel and confounding bright pathologies, as shown in Fig. 1. Moreover, most of these existing techniques segment optic disc and cup independently and the strong shape and location priors of neuroretinal rim (as shown in Fig. 1) cannot be fully used. Finally, most of them try to eliminate the interference of blood vessel with an image preprocess of vessel removal which turned out to be a very hard task in practice.

To deal with the above-mentioned challenges of automatic segmentation of optic disc and cup, we propose to address this task by introducing graph cut which is an effective combinatorial optimization technique [8] and incorporating priors not only on shape, location and geometric interactions of optic disc and cup, but also on rim thickness. Our algorithm is reliable and robust not only because graph cut offers a globally optimal solution but also due to the incorporated priors which can effectively reduce the search space of feasible segmentations. Different from previous work, the interference of blood vessel can be accounted for in the segmentation process by modeling the vessel intensity information instead of removing vessels with a preprocess.

We describe a fairly general energy function which integrates a set of model-specific visual cues, contextual information and topological constraints to include a wide range of priors. First, boundaries of both “cup” and “disc” are nearly circular around optic disc center and their radii can be a prior approximated. Second, “cup” is contained in “disc”. Third, the minimum rim thickness can also be a prior estimated for the given data set. Our energy function is submodular and can therefore be formulated as a s-t cut problem and minimized efficiently via graph cut (min-cut/max-flow algorithm) [8].

We evaluate the performance of our algorithm not only by comparing with manual delineations of optic disc and cup on a rigorous pixel-by-pixel basis, but also by comparing the algorithm’s assessments on a set of ONH parameters from color fundus photograph against the corresponding evaluations generated by the merchant-provided software in the Optical Coherence Tomography (OCT) machine. Our results show that the proposed algorithm outperforms two state-of-the-art segmentation methods and demonstrates strong correlations with the merchant-provided software in OCT imaging.

2 Method

Upon the location of optic disc center is known, our segmentation task amounts to searching regions of optic disc and cup which boundaries are circle-like, around the optic center and with radii that can be a priori approximated.

2.1 Simultaneous Segmentation of Optic Disc and Cup

Our optic disc and cup segmentation framework divides a rectangle region around the optic disc center into parts of cup, disc and background. It relies on the powerful graph cut algorithm [8] and allows for spatial interaction of those segmented parts and the incorporation of priors on location and shape of and distance between disc and cup boundaries. Our framework results in a submodular energy minimization problem and can therefore generate a global optimization.

Let $\mathcal{P}$ be the set of pixel indices and $\mathcal{L}$ be the set of region indices. We set $\mathcal{L}=\{c,d\}$ where the set elements denote “cup” and “disc”, respectively. We then define binary variables ${\mathbf{x}}_p^l\in {\mathbb{B}}^{\mathcal{L}\times \mathcal{P}}$ where $l\in \mathcal{L}$ and $p\in \mathcal{P}$ index x, $\mathbb{B}=\{0,1\}$ and x_p is a binary vector of all variables corresponding to p. Obviously, x_p is in length of $\mid \mathcal{L}\mid$ where ∣·∣ denotes the number of elements in a set. If pixel p is in region l, then we have ${\mathbf{x}}_p^l=1$. If x_p = 0, pixel p is in the region of “background”.

The proposed energy function to be minimized via graph cut is defined as

$$\begin{array}{cc}\hfill E\left(\mathrm{x}\right)=\sum _{p\in \mathcal{P}}{D}_p\left({\mathrm{x}}_p\right)+\sum _{p,q\in {\mathcal{N}}_{\nu }}& \left(V_{p,q}^c\left({\mathbf{x}}_p^c,{\mathbf{x}}_q^c\right)+V_{p,q}^d\left({\mathbf{x}}_p^d,{\mathbf{x}}_q^d\right)\right)\hfill \\ \hfill & +\sum _{p,q\in {\mathcal{N}}_o}{W}_{p,q}^{c,d}\left({\mathbf{x}}_p^c,{\mathbf{x}}_q^d\right)\hfill \end{array}$$ (1)

where the three ∑ components contain data terms, smoothness terms and region interaction terms [9], respectively, and are to be defined in the following.

2.2 Data Terms

The data term D_p(x_p) in Eq. (1) is constructed from the probability of each pixel p belonging to any of the regions r ∈ {“cup”, “disc”, “background”}. For each region, we train a Gaussian Mixture Model (GMM) and use it to decide a posterior probability with the following equation

$$P(r\mid I_p)=\frac{P(I_p\mid {\Theta }_r)P\left({\Theta }_r\right)}{P\left(I_p\right)}\approx \sum _{k=1}^K{\alpha }_k^{r}G(I;{\mu }_k^r,{\Sigma }_k^r)$$ (2)

In Eq. (2), P(I_p∣Θ_r) is the probability density function of each region r, P(Θ_r) is the prior probability, and P(I_p) is merely a scaling signatures factor and can be neglected.

I_p is the fundus color vector of pixel $p\in \mathcal{P}$. Region r is assumed to contain K (empirically set to 4) components and $G(I;{\mu }_k^r,{\Sigma }_k^r)$ is the Gaussian probability function with mean value ${\mu }_k^r$ and covariance matrix ${\Sigma }_k^r$. α_k are positive weights of the component k and ${\sum }_{k=1}^K{\alpha }_k=1$. The parameter list Θ^r = {α₁, μ₁, ∑₁, ⋯ , α_K, μ_K, ∑_K} defines a particular GMM probability function. The parameters in Θ^r are estimated with the Expectation Maximization (EM) algorithm (an iterative method for calculating maximum likelihood distribution parameters) and a set of manually labeled color fundus ONH photographs.

With the posterior probability of Eq. (2), D_p(x_p) in Eq. (1) is defined in Table. 1, which is apparently submodular [8] considering the fact that

$$D_p\left(\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \end{array}\right]\right)+D_p\left(\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \end{array}\right]\right)<D_p\left(\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 1\hfill \end{array}\right]\right)+D_p\left(\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 0\hfill \end{array}\right]\right).$$

More importantly, the ∞ value in Table. 1 is specified in order to prohibit the corresponding configuration (i.e. a “cup” pixel is not in “disc” region) of x_p. In other words, this defination can help to guarantee the truth that “cup” region is contained in the “disc” region.

Table 1.

Definition of data term D_p(x_p) and region interaction term $W_{p,q}^{c,d}\left({\mathbf{x}}_p^c,{\mathbf{x}}_q^d\right)$ in Eq. (1).

${\mathbf{x}}_p^c$	0	0	1	1
${\mathbf{x}}_p^d$	0	1	0	1
Dp(xp)	–log P(“background”|Ip)	–log P(“disc”|Ip)	∞	–log P(“cup”|Ip)
${\mathbf{x}}_p^c$	0	0	1	1
${\mathbf{x}}_q^d$	0	1	0	1
$W_{p,q}^{c,d}\left({\mathbf{x}}_p^c,{\mathbf{x}}_q^d\right)$	0	0	∞	0

2.3 Smoothness Terms

The smoothness terms in Eq. (1) are enforced only on neighboring pixels which are defined on a nearest neighbor grid connectivity ${\mathcal{N}}_{\nu }$ (e.g. 4-connectivity in a 2D space). Specifically, for each $l\in \mathcal{L}$, this pairwise term is defined as a Pott’s-like model:

$$V_{p,q}^l\left({\mathbf{x}}_p^l,{\mathbf{x}}_q^l\right)=B_{p,q}\cdot {\delta }_{{\mathbf{x}}_p^l\ne {\mathbf{x}}_q^l}$$ (3)

where

$${\delta }_{{\mathbf{x}}_p^l\ne {\mathbf{x}}_q^l}=\{\begin{array}{cc}1\text{if}{\mathbf{x}}_p^l\hfill & \ne {\mathbf{x}}_q^l\hfill \\ 0\text{if}{\mathbf{x}}_p^l\hfill & ={\mathbf{x}}_q^l\hfill \end{array}\phantom{\}}$$ (4)

is a Kronecker delta function and B_p,q comprises the “boundary” properties of “cup” and “disc”. We define B_p,q as

$$B_{p,q}\propto \exp \left(-\frac{{\Vert I_p-I_q\Vert }^2}{2{\sigma }^2}\right)\cdot \frac{\mid R_{p,q}-R_l\mid }{\mid R_{p,q}-R_l\mid +∊}$$ (5)

where σ is an adjusting parameter (empirically set to 4), R_p,q denotes the average value of distances of pixels p and q to the optical center, R_l represents a radius value a priori approximated based on our knowledge of ONH anatomy and the photograph resolution (we empirically set R_c = 210 pixels and R_d = 345 pixels), and ∊ is an adjusting parameter (empirically set to 0.4). In Eq. (5), the left term is large when the pixels p and q are similar in their color and close to zero when the two are very different, resulting in a lot penalization for label discontinuities between pixels of similar colors. The right term encourages the detected boundary to appear in the vicinity of a circle with an a priori known radius of “cup” or “disc”. Parameter ∊ controls how far the detected boundary can appear away from the a priori known boundary.

2.4 Region Interaction Terms

The region interaction terms $W_{p,q}^{c,d}\left({\mathbf{x}}_p^c,{\mathbf{x}}_q^d\right)$ in Eq. (1) are employed on all pairs (denoted by ${\mathcal{N}}_o$) of pixels between which the distance is below a a priori known value. This value is assigned to the thinnest rim thickness (empirically set to 15 pixels) in our experiments. As defined in Table. 1, these region interaction terms can help to enforce two constraints that the “disc” region contains the “cup” region and that the rim thickness is above the a priori known value. Inclusion of this region containment and a hard uniform margin (between “cup” and “disc” boundaries) into graph cut are also explained in [9].

2.5 Optimization

As defined in Table. 1 and Eq. (3), the energy function E(x) in Eq. (1) is obviously submodular [8] and therefore can be formulated as a s-t cut problem and minimized efficiently via graph cut (min-cut/max-flow algorithm) [8]. A local minimum lying within a multiplicative factor of the global minimum can be obtained with the α-expansion algorithm [8].

2.6 Detection of Optic Disc Center

Detection of optic disc center is necessary because the shape and location priors applied to our segmentation scheme require the location of optic disc center. We employ an automatic method based on template matching followed by a false positive removal with a directional matched filtering similar to [6], which are based on the fact that optic disc appears in the fundus image as a brighter circular area and that the main vessels originate from the optic disc center, respectively. Technical details are omitted for economy. In our experiments, manual adjustments were applied to the obviously wrong detections.

3 Results

3.1 Data

Stereoscopic ONH photographs, taken by a simultaneous stereo fundus camera (Zeiss 30°), of bilateral eyes of 30 progressed glaucoma subjects and 32 control subjects were used (totally 248 color fundus photographs), with size of 239× 2042 pixels and resolution of 2.6 × 2.6μm². HD-OCT (Cirrus; Carl Zeiss Meditec. Inc.,) scans were also acquired with the ONH parameters of disc area, rim area, cup area and cup-to-disc (C/D) area ratio generated by the Cirrus HD-OCT software (version 5.0). Optic cup and disc margins were manually delineated by a trained rater with the stereo disc photographs of randomly chosen 15 glaucoma subjects and 16 control subjects.

3.2 Experiments

We first compared the optic disc and cup segmentation of our algorithm with the active contour model [4] and the original graph cut algorithm [8]. For the original graph cut algorithm, we removed all location and shape priors of optic cup and disc and only kept its original smoothness term. Mean±STD values of the average distance between the automatically detected cup and disc margins and the subjective assessed margins (of the 15 glaucoma subjects and 16 control subjects) were computed and listed in Table 2. We also put the segmentation results for two example images in Fig. 2.

Table 2.

Statistics of average distance between the subjectively assessed optic cup and disc margins and the automatically detected margins by our algorithm (OUR), the active contour model (ACM) [4] and graph cut (GC) [8] from photos of 16 normal controls and 15 glaucoma subjects.

Mean±STD	Normal			Glaucoma
	OUR	ACM	GC	OUR	ACM	GC
Cup	*8.2 ± 6.1	42.9 ± 28.5	31.3 ± 22.9	*10.5 ± 7.7	55.6 ± 34.4	48.1 ± 29.6
Disc	*14.5 ± 12.4	46.2 ± 20.0	50.7 ± 28.3	*16.9 ± 12.8	60.8 ± 44.0	59.2 ± 30.7

^*indicates the best performance value.

Fig. 2.

ONH images (only a rectangle region around optic disc is shown) and margins of cup (blue) and disc (black) manually assessed and automatically detected by our algorithm (OUR), the active contour model (ACM) [4] and graph cut (GC) [8].

From the results in Table 2 and Fig. 2, we had several findings. First, our algorithm outperformed the active contour model and the original graph cut for segmentation of both optic cup and disc. Second, all automatic algorithms performed better on normal controls than glaucoma subjects. Third, graph cut was comparable with the active contour model regarding optic disc segmentation but superior in sense of optic cup segmentation.

Failure of the automatic algorithms to detect the cup and disc margins properly was defined as when the average distance between the automatically detected margin and the subjective assessed margin was larger than 34 pixels (5% of the average disc diameter). We found that our algorithm failed only for one glaucoma subject, and in contrast, the active contour model failed for 10 glaucoma subjects and 9 normal controls while graph cut failed for 8 glaucoma subjects and 6 normal controls.

The results of the ONH parameters (including the subject defined as failure) generated by our algorithm were compared with the results produced by the merchant-provided software of the OCT machine. The corresponding Mean ± STD values of each parameter and the correlation coefficient (r) and p-value (p) of Pearson correlation between our algorithm and the OCT machine were listed in Table. 3. We can see that all ONH measurements based on OCT machine-defined margins showed a high correlation with our algorithm which uses fundus photographs. For each eye, the average of our algorithm’s evaluations on the stereoscopic pair was used as the final result.

Table 3.

Comparison of ONH measurements generated by the merchant-provided software of the OCT machine and our algorithm regarding area (mm²) of disc, rim and cup and C/D ratio.

Parameter	Normal				Glaucoma
	OCT*	Algorithm*	r†	p†	OCT*	Algorithm*	r†	p†
Disc area (mm2)	2.50 ± 0.54	2.41 ± 0.39	0.88	0	2.97 ± 0.88	2.94 ± 0.86	0.86	4e-13
Rim area (mm2)	1.71 ± 0.69	1.64 ± 0.67	0.86	2e-12	1.45 ± 0.75	1.46 ± 0.78	0.82	7e-8
Cup area (mm2)	0.78 ± 0.51	0.77 ± 0.63	0.85	3e-19	1.52 ± 0.82	1.48 ± 0.79	0.80	9e-5
C/D Ratio	0.31 ± 0.14	0.32 ± 0.23	0.85	5e-17	0.51 ± 0.20	0.50 ± 0.34	0.81	3e-6

^*Mean ± STD;

^†Pearson correlation

4 Conclusion and Future Work

Inspired by recent image segmentation methods that can combine certain shape priors or region/boundary interactions [9], we propose a fairly general energy function that can naturally fit into a global optimization framework with graph cut for the automatic segmentation of optic cup and disc from color fundus photographs. Our main contribution lies in the introduction of a set of priors obtained from the fundus anatomical characteristics of optic nerve head, which include the location and shape priors of optic cup and disc, the thinnest thickness of rim and the fact that “disc” contains “cup”. These priors can help to greatly reduce the search space of feasible solutions, and together with the effective optimization of graph cut, generate reliable and robust segmentation results.

Our approach incorporates shape priors in a different fashion from several existing techniques which are based on shape statistics (e.g. the deformable segmentation [10, 11] and the Active Shape Model [12]). Shape statistics are mostly obtained from a set of training images. In contrast, our approach encodes the prior knowledge (on contextual information or geometric interactions) in the algorithm directly and is free from any training process.

Our future work would include tests on a larger data set and more evaluations on the automatic detection of optic disc center.