Simultaneous Segmentation of Retinal Surfaces and Microcystic Macular Edema in SDOCT Volumes

Abstract

Optical coherence tomography (OCT) is a noninvasive imaging modality that has begun to find widespread use in retinal imaging for the detection of a variety of ocular diseases. In addition to structural changes in the form of altered retinal layer thicknesses, pathological conditions may also cause the formation of edema within the retina. In multiple sclerosis, for instance, the nerve fiber and ganglion cell layers are known to thin. Additionally, the formation of pseudocysts called microcystic macular edema (MME) have also been observed in the eyes of about 5% of MS patients, and its presence has been shown to be correlated with disease severity. Previously, we proposed separate algorithms for the segmentation of retinal layers and MME, but since MME mainly occurs within specific regions of the retina, a simultaneous approach is advantageous. In this work, we propose an automated globally optimal graph-theoretic approach that simultaneously segments the retinal layers and the MME in volumetric OCT scans. SD-OCT scans from one eye of 12 MS patients with known MME and 8 healthy controls were acquired and the pseudocysts manually traced. The overall precision and recall of the pseudocyst detection was found to be 86.0% and 79.5%, respectively.

1. INTRODUCTION

Spectral-domain optical coherence tomography (SDOCT)^1,2 has begun to find widespread use in the diagnosis and management of a variety of ocular diseases. This noninvasive imaging modality uses near-infrared light to image the retina and allows for the quantitative analysis of retinal structures. SDOCT has also begun to find application in studying neurological disorders such as multiple sclerosis (MS), where optic neuropathy has been observed in a significant number of patients.³ In addition to decreasing estimates of the retinal nerve fiber layer (RNFL) and ganglion cell layer (GCL) thicknesses, pseudocysts or microcystic macular edema (MME) may occur in the eyes of about 5% of MS patients.⁴ They occur most frequently in the inner nuclear (INL), outer plexiform (OPL), and outer nuclear layers (ONL). However, these pseudocysts are not limited to MS⁵ and has also been found to be associated with other causes.^6,7 Figure 1 shows an example of a scan obtained from an MS subject, with the MME overlaid on the scanning laser ophthalmoscope fundus image and a B-scan from an SDOCT image. While the underlying cause of these cysts has not been identified, their presence has been correlated with disease severity in MS,⁸ thus warranting accurate identification and tracking of these structures.

Figure 1.

Visualization of the pseudocysts seen in an SDOCT volume obtained from an MS subject on (a) the scanning laser ophthalmoscope fundus image, and (b) a B-scan from the parafoveal location indicated by the white arrow.

Previously, we proposed two separate approaches for the segmentation of the retinal layers and the MME, where the surfaces were segmented using a graph-theoretic approach⁹ and the MME were detected using voxel classification,^10,11 respectively. A graph-based approach was previously proposed by Chen et al. for the segmentation of larger fluid regions in the outer retina.¹² This method however, relies on a preprocessing stage at which the 2-D locations of the regions are detected. Moreover, this method did not perform well in the case of small cysts, as is usually the case with MME. More recently, a combined machine-learning and graph-cut approach was proposed for the segmentation of fluid filled regions in the retina,¹³ however the retinal surfaces were not simultaneously segmented in this formulation.

Herein, we propose a graph-theoretic approach for the simultaneous segmentation of retinal surfaces and pseudocysts. These structures are not independent and the simultaneous segmentation of the retinal surfaces and MME could help prevent errors, such as the detection of MME in unexpected locations or errors in the surface segmentation. This is done by combining the closed set multiple surface segmentation approach^14,15 and multi-region graph-cuts.^16,17 The final graph is solved using a minimum s-t cut, thereby ensuring the global optimality of the final solution. Previously, Chen et al.¹² described a graph-based method for the detection of large fluid filled regions in the outer retina. However, this method requires a 2-D footprint as a prior, while the proposed method does not require an estimate of the location of the pseudocysts. Furthermore, the absence of the cysts will not affect the retinal surface segmentation accuracy. The overall precision and recall of pseudocyst detection was found to be 86.0% and 79.5%, respectively. The surface segmentation frequently seen errors, specifically in the segmentation of the NFL-GCL and the GCL-INL boundaries, were also seen to improve when using this combined graph-theoretic framework.

2. METHOD

The simultaneous segmentation of multiple retinal surfaces and the pseudocysts is achieved by combining two graph-based approaches – namely graph-cuts¹⁷ and the minimum cost closed set formulation.^14,18 The graph-cuts allow for the segmentation of multiple foreground structures, and while this may not be suitable for the segmentation of layered structures, it is well suited to the segmentation of disjointed structures such as pseudocysts. On the other hand, the minimum-cost closed set formulation allows for the simultaneous segmentation of multiple surfaces, and has proven to be extremely useful in the segmentation of retinal surfaces in SDOCT images.^9,15,19–21 Both of these graph-theoretic formulations are solved using a minimum s-t cut, and thus, it is possible to link the graphs whereby multiple surfaces and pseudocysts can be segmented simultaneously, while ensuring a globally optimal solution. The graph structure and the cost functions used in this formulation are described in further detail below.

2.1 Graph-theoretic Formulation

The graph for the minimum-cost closed set assumes that the surface of interest $\mathcal{S}$ intersects each column of the image $\mathcal{I}\left(x,y,z\right)$ of size X × Y × Z, at exactly one location, such that $\mathcal{S}\left(x,y\right)=z$ with $(x,y,z)\in \mathcal{S}$. The topology of the surface is constrained by an upper bound on the expected change in the surface at adjacent columns p, q (given a predefined neighborhood $\mathcal{N}$, e.g., a 4-neighbor relationship) such that

$$-{\mathrm{\Delta }}_{p,q}^u\le \mathcal{S}(p)-\mathcal{S}(q)\le {\mathrm{\Delta }}_{p,q}^l,$$ (1)

where ${\mathrm{\Delta }}_{p,u}^u$ and ${\mathrm{\Delta }}_{p,u}^l$ represent the upper and lower bounds on the expected change in the surface between two adjacent columns p and q, respectively. These values can also be learned a priori from a training set and vary by column location and surface.²² Similarly, the expected distances between surfaces i and j can also be constrained such that

$${\delta }_{(i,j)}^{min}<{\mathcal{S}}_j-{\mathcal{S}}_i<{\delta }_{(i,j)}^{max},$$ (2)

where ${\delta }_{(i,j)}^{min}$ and ${\delta }_{(i,j)}^{max}$ represent the expected distances between surfaces i and j, respectively.

Figure 2(a) illustrates the incorporation of these surface topology and interaction constraints. The nodes are also connected to a source and sink using directed edges.¹⁴ The minimum-cost closed set is found by finding the minimum s-t cut on this graph, and envelope of the closed set of each subgraph gives us the final multiple segmented surfaces. The key aspect of this graph is the closed set structure that ensures the continuity of the segmented surfaces.

Figure 2.

Illustrations of (a) the minimum-cost closed set formulation proposed¹⁴ for the segmentation of multiple surfaces, and (b) the graph-cut¹⁷ image segmentation, and (c) the combined graph used for the segmentation of multiple retinal surfaces and the pseudocysts.

In graph-cuts,¹⁷ each image voxel is represented by a node, which is connected by weighted edges, referred to as terminal links (t-links), to the source S and the sink T. Node $v\in \mathcal{I}(x,y,z)$, for instance, is connected to the S by the edge E(S, v) = C_S(v), where C_S(v) indicates the cost associated with the pixel belonging to the foreground (or belonging to the source node set). Similarly, the node n is also connected to T by the edge E(v,T) = C_T(v), where C_T(v) indicates the cost associated with the pixel belonging to the background (or belonging to the sink node set). Figure 2(b) shows an example with the nodes of the image connected to the source S and the sink T. Adjacent nodes v_i and v_j within a specified neighborhood $\mathcal{N}$ are also connected to each other by neighborhood links (n-links), E(v_i, v_j) = B(v_i, v_j), where B(v_i, v_j) represents the boundary cost associated with the edge E(v_i, v_j) belonging to the minimum cut. Figure 2(b) shows a simple two-column example of such a graph, with the t-links shown in green and the n-links shown in black.

This formulation can also be extended to geometric configurations involving more than one foreground structure, such as when one structure is contained within another. Delong et al.¹⁶ described a multi-region framework, where infinity-weighted arcs are introduced to enforce geometric constraints such as inclusion or exclusion criteria among interacting foreground structures. Subgraphs are created for each foreground structure and are connected in order to enforce the a priori constraints. For instance, consider voxels vⁿ and v^m that belong to regions n and m, respectively. If the region n is contained with region m, then setting the weight on edge E(vⁿ,v^m) = ∞ will prevent the edge E(vⁿ,v^m) from being a part of the minimum cut.

We can formulate the combined MME and layer segmentation into this geometric constraint framework by connecting the MME and boundary sub-graphs such that the MME is constrained to lie below a particular boundary and the ordering of consecutive boundaries is enforced. Figure 2(c) shows one column (orange) from Surface 1 subgraph, corresponding to the surface above the pseudocysts, Surface 2 (blue) lies below Surface 1 and is connected using interaction constraints. The pseudocyst subgraph (purple) is connected to Surface 1, but there are no interaction constraints between the pseudocysts and Surface 2. Note, the neighborhood can also be defined to enforce a minimum distance between Surface 1 and the pseudocysts. The infinity weighted arcs (solid black) will not be part of the cut, and thus, enforce the inclusion and boundary distance constraints.

Figure 3 shows the steps involved in creating the graph used to simultaneously segment the retinal surfaces and the pseudocysts. In the previously described approach that segmented nine retinal surfaces, the inner retinal surfaces,⁹ namely the bottom of the RNFL, GCL, INL, and OPL are segmented simultaneously, after the ILM and the outer retinal surfaces are detected. These steps remain unchanged in the current method, and the subgraphs for these four surfaces are created using the minimum-cost closed set framework. The pseudocyst subgraph is constructed using the graph-cut formulation. Moreover, since the pseudocysts are known to occur in either the INL, OPL, or ONL, this subgraph is linked to the INL such that the pseudocysts are included within this surface closed set. This prevents any pseudocysts from being detected above this surface.

Figure 3.

Flowchart illustrating the simultaneous segmentation of 4 inner retinal surfaces and the pseudocysts using the proposed graph-theoretic approach.

2.2 Cost Function Design

The total cost of the set of n surfaces and the pseudocysts can be represented as C_T = C_surf + C_cyst, where C_surf represents the cost of the n surfaces and C_cyst represents the cost associated with the pseudocysts. In particular,

$$C_{\text{surf}}={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{(x,y,z)|z={\mathcal{S}}_i(x,y)}{c}_i(x,y,z)}},$$ (3)

where c_i(x, y, z) is the cost associated with voxel (x, y, z) lying on surface i. This cost function was designed using a machine-learning approach,⁹ where a random forest²³ was trained on an independent set of manually delineated scans, in order to create probability maps for the retinal surfaces.

The second term C_cyst was designed as follows:

$$C_{\text{cyst}}={\displaystyle \sum _{v\in F}c(v)+\alpha }{\displaystyle \sum _{v_i,v_j\in \mathcal{N}}B(v_i,v_j)},$$ (4)

where c(v) is the cost of voxel v belonging to the foreground F, and B(v_i, v_j) is the boundary cost associated with the voxel v_i lying on the boundary of the foreground object.

The pseudocyst region cost was also designed using a random forest classifier,^10,11 where c(v) = ηe^−p, where p reflects the probability value obtained from the random forest. The boundary cost was computed as $B(v_i,v_j)=e^{\frac{-{(I(v_i)-I(v_j))}^2}{2{\sigma }^2}}$, where I(v) is the intensity of voxel v. The parameters η, α, and σ were empirically chosen and set to 180, 20, and 130, respectively.

3. EXPERIMENTAL METHODS & RESULTS

Twelve volumetric SDOCT scans were acquired from nine MS patients that presented with MME and 8 healthy controls (HC).¹¹ The macular raster scans were acquired on a Spectralis OCT system (Heidelberg Engineering, Heidelberg, Germany) and imaged a region approximately 6mm×6mm×2mm. The volume consisted of 49 B-scans, 1024 A-scans per B-scan, and 496 voxels per A-scan. A leave-one-subject-out procedure was used to train the pseudocyst random forest and test the remaining eight subjects. The random forest for the surface segmentation was trained on a completely independent set of scans.⁹ The pseudocyst detection precision, recall, and F-measure were computed as shown below.

$$\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}},\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}},\text{and}\mathrm{F}-\text{measure}=\frac{2\times \text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}},$$ (5)

where, TP is the number of pseudocysts correctly found, FP is the number of false positives, and FN is the number of false negatives or pseudocysts missed. (A pseudocyst is marked as a true positive if the automated method and the manual segmentation overlap.) The pseudocysts were also segmented using the voxel classification approach¹¹ and the results from the two approaches were compared.

The twelve MS scans were divided into two groups—low density and high density scans—based on the number of cysts present. On average, the low density set contained approximately 16 cysts per scan, while the high density set contained 346 per scan. The results of the MME detection are shown in Table 1. The previously proposed voxel classification method^11,24 showed an overall precision, recall and, F-measure of 86.2%, 77.5%, and 81.6% respectively. The proposed graph-theoretic method, on the other hand, showed an overall precision, recall and, F-measure of 86.0%, 79.5%, and 82.6% respectively. Figure 4 shows an example of a B-scan with the detected pseudocysts. Whereas for the healthy control cohort of eight scans, the voxel classification approach erroneously detected two pseudocysts while the proposed method only detected one.

Table 1.

Summary of pseudocysts detection accuracy statistics in the MS patients and the HC scans.

Method	Metric	Low Density	High Density	Healthy Controls
	# Cysts	98	2076	0
Voxel Classification11,24	# Cysts detected	43	1641	2
	Precision	0.896	0.861	–
	Recall	0.439	0.790	–
	F - measure	0.589	0.824	–
	# Cysts detected	43	1685	1
Proposed Method	Precision	0.935	0.849	–
	Recall	0.439	0.812	–
	F - measure	0.597	0.830	–

Figure 4.

A slice from an MS patient scan showing (a) the original B-scan, (b) the manual tracings and (c) the automated result obtained using the proposed method.

The surfaces using the proposed method and the alternative automated method were qualitatively examined. No differences were noted in regions free of pseudocysts; however, in certain locations, the thin nerve fiber layer combined with the presence of the disruptive pseudocysts caused the surface segmentation algorithm⁹ to fail, while the proposed method was able to correct the surface segmentation in these location. Figure 5 shows an example of one of these problem areas.

Figure 5.

A slice from an MS patient scan showing (a) the original B-scan, (b) the segmentation result of the inner surfaces obtained using the previously described graph method⁹ and (c) the segmentation result obtained by the proposed method that simultaneously segments the surfaces and the pseudocysts. The red arrows indicate areas where the segmentation of the RNFL-GCIP and the GCIP-INL boundaries failed, even cutting through MMEs at certain locations. However, the use of the proposed method where the MMEs and the surfaces are simultaneous segmented prevents such errors.

4. CONCLUSION

Microcystic macular edema are an elusive and ill-understood phenomenon, occurring within a small percentage of the MS population. While these cysts are known to occur within specific retinal layers, their presence is hard to estimate. The proposed method incorporates this crucial information and simultaneously segments the retinal layers and any cysts that may be present in the images. The biggest advantage of using the simultaneous formulation is the ability to prevent surface segmentation errors that may occur in the presence of a large number of cysts as shown in Fig. 5. Moreover, the absence of cysts will also not affect the surface segmentation accuracy.

The proposed method’s graph structure utilizes an additional subgraph for the pseudocysts which does increase the memory requirements and runtime of the method as compared to the graph-based retinal layer segmentation approach.^9,15 Furthermore, graph-cuts requires that the data cost be submodular, and thus certain geometric configurations cannot be optimized using the multi-region framework. For instance, if a foreground structure is included within another while also being excluded from a third, only one of these constraints, either the inclusion or the exclusion, can be enforced. Thus, in the method described here, the pseudocysts can be explicitly defined as structures that exist below the IPL-INL boundary, but a lower bound cannot be simultaneously set.

In our future work, we intend to explore the possibility of enforcing both, the upper and lower boundary constraints, perhaps using α-expansion.²⁵ Additionally, we expect that the ability to accurately monitor and track pseudocysts in MS patients will provide insight into the cause and ultimate impact of these cysts on visual function.