Pixel-wise segmentation of severely pathologic retinal pigment epithelium and choroidal stroma using multi-contrast Jones matrix optical coherence tomography

Abstract

Tissue segmentation of retinal optical coherence tomography (OCT) is widely used in ophthalmic diagnosis. However, its performance in severe pathologic cases is still insufficient. We propose a pixel-wise segmentation method that uses the multi-contrast measurement capability of Jones matrix OCT (JM-OCT). This method is applicable to both normal and pathologic retinal pigment epithelium (RPE) and choroidal stroma. In this method, “features,” which are sensitive to specific tissues of interest, are synthesized by combining the multi-contrast images of JM-OCT, including attenuation coefficient, degree-of-polarization-uniformity, and OCT angiography. The tissue segmentation is done by simple thresholding of the feature. Compared with conventional segmentation methods for pathologic maculae, the proposed method is less computationally intensive. The segmentation method was validated by applying it to images from normal and severely pathologic cases. The segmentation results enabled the development of several types of en face visualizations, including melano-layer thickness maps, RPE elevation maps, choroidal thickness maps, and choroidal stromal attenuation coefficient maps. These facilitate close examination of macular pathology. The melano-layer thickness map is very similar to a near infrared fundus autofluorescence image, so the map can be used to identify the source of a hyper-autofluorescent signal.

1. Introduction

Optical coherence tomography (OCT) [1] is a powerful imaging modality, particularly in the field of ophthalmology, because it offers cross-sectional and three-dimensional (3-D) tomography with high contrast. OCT reveals the retinal layers, by measuring backscattering intensity, making OCT-based diagnoses integral to current-day ophthalmology. As the popularity of OCT increases, the demand for automatic and quantitative analysis of retinal morphology, such as automatic area segmentation of specific tissues or pathologic regions [2, 3] and/or layer segmentation of the retinal layers [4–7], also increases.

Examples of automatic segmentation methods include the peak-detection method [4], graph theory and dynamic programing [5], the 3-D graph method [6], and the active contour method [7]. These conventional methods work well in OCT images of tissue without severe structural abnormalities. However, in heavily damaged or degenerated tissue, the accuracy of the segmentation becomes inadequate [4, 8]. Some methods have overcome this limitation, but they generally require an iterative process and are time-consuming [9–12].

OCT angiography (OCTA) [13–16] and polarization-sensitive OCT (PS-OCT) [17–19] are functional extensions of OCT that measure the blood flow and polarization properties of the sample, respectively. Multi-functional Jones matrix OCT (JM-OCT) is a comprehensive version of OCTA and PS-OCT that measures conventional scattering OCT, OCTA, phase retardation/birefringence and degree-of-polarization-uniformity (DOPU) tomographies [20], simultaneously [21–25]. Although such images have been used for subjective observations, only a few methods for morphological analysis using JM-OCT have been reported [2, 18, 20].

Since JM-OCT provides multiple images with different contrasts, it should provide more information than conventional OCT does, and the increased information should enable more-convenient and less-time-consuming segmentation methods. In addition, if a segmentation method were designed to use only the multiple-contrast information without morphological information, it would become a robust segmentation method not only for normal to moderately pathologic cases, but also for cases with extensive morphological changes.

In this paper, we demonstrate a new segmentation method that exploits multiple types of images obtained by JM-OCT, including the attenuation coefficient (AC), DOPU, and OCTA. In the segmentation process, we synthesized “features” from the above-mentioned multi-contrast JM-OCT images, and designate them as primary optical features. A synthesized feature is designed to be sensitive to particular tissues of interest, denoted as target tissues, so applying a single threshold to the synthesized feature enables the segmentation of the target tissue. This method is not a boundary-delineation segmentation, but a pixel-wise classification that labels each pixel as either a target or non-target tissue. This method is also designed not to depend upon morphological information, so it is equally applicable to OCT images from normal and severely structurally damaged retinas. In addition, this method does not include an iterative process and thus can be computed quickly.

Furthermore, we create several types of en face maps, by combining the multiple contrasts from JM-OCT with the segmentation results from retinal pigment epithelium (RPE) and choroidal stroma. The en face maps, including a melano-layer thickness map, RPE elevation map, choroidal thickness map and average choroidal stromal AC map, facilitate the identification of pathologic retinal structures. We evaluate the clinical utility of this new method by applying it to several clinical cases, which include pigment epithelial detachment (PED), geographic atrophy (GA), and exudative age-related macular degeneration (AMD).

2. Jones matrix optical coherence tomography system

Two custom-made swept-source-based JM-OCT prototypes with similar specifications were used to obtain multi-contrast images. The first system, which is a laboratory prototype for testing normal subjects [23], uses a wavelength sweeping laser with a scanning frequency of 100 kHz, a center wavelength of 1048 nm, a sweeping wavelength range of 123 nm, and a measured depth resolution of 6.2 μm in tissue. The other system is a clinical prototype that is used for evaluating patients [25]. It uses a wavelength sweeping laser with a scanning frequency of 100 kHz, a center wavelength of 1060 nm, a sweeping wavelength range of 110 nm, and a measured depth resolution of 6.6 μm in tissue. Both systems scan an area of 6 mm × 6 mm of retina in 6.6 s with a horizontal fast raster scan protocol. The volumetric data consist of 512 A-lines (horizontal) × 256 vertical positions × 4 repetitive scans at each vertical position.

JM-OCT simultaneously measures a set of multi-contrast images including backscattering intensity, AC, OCTA, DOPU, and birefringence image. The scattering intensity is calculated by coherent composition [23, 24] and is similar to conventional OCT. The AC is calculated from the scattering intensity using a model-based reconstruction method by Vermeer et al. [26] with an extension for polarization diversity detection (Eqs. (12) and (14) of Ref. [27]). By employing the AC, we can regulate the OCT intensity signal, which is susceptible to opacity of ocular media, such as cataracts, and to shadowing by abnormal hyper-scattering materials in the retina. OCTA is computed by the complex Jones matrix correlation method with noise correction [28]. Noise-corrected OCTA is a complex decorrelation without a noise offset. While OCTA is a decorrelation, it ranges not only between [0.0, 1.0] but also takes negative values or values larger than 1.0 to cancel the noise offset statistically. DOPU is known to be low if melanin exits [29], therefore it selectively contrasts melanin-containing tissues such as the RPE [20]. In our particular implementation, DOPU is computed with Makita’s noise correction [30], and a 3 pixel (transverse) × 3 pixel (depth) kernel. Similar to the OCTA with noise-offset correction, this noise-corrected DOPU also not only ranges between [0.0, 1.0] but also takes negative values or values of more than 1.0. The birefringence was computed by combining depth-localized Jones matrix analysis [31] and subsequent maximum a posteriori (MAP) birefringence estimation [32, 33]. The size of the depth-localized Jones matrix was 9 pixels (34 μm) and the kernel size of the MAP birefringence estimation was 2 pixels (transverse) × 2 pixels (depth). Please note that the birefringence is only used to help understand the retinal pathology, but is not used for the segmentation.

3. Multi-contrast segmentation

3.1. Segmentation method

The multi-contrast segmentation method is based on the idea of feature engineering. In this segmentation method, a “feature”, which is sensitive to a specific target tissue, is synthesized by combining the multi-contrast images. Once the feature has been synthesized, tissue classification is achieved by applying a simple threshold to the feature. This tissue classification is pixel-wise segmentation and does not use morphological information. And, hence, it can easily be applied to cases with severe morphological abnormality.

Our method is designed for the segmentation of RPE and choroidal stroma. The feature synthesis for these tissues depends upon prior knowledge about the contrast properties of these tissues in the multi-contrast images (as summarized in Table 1). For example, both the RPE and choroidal stroma contain melanin. Hence they should show low DOPU, while the inner retina should show high DOPU. Choroidal stroma consists of dense vasculature, hence its OCTA signal should be high, whereas the RPE has no OCTA signal. The choroidal lumen is expected to have low OCTA signal because of signal washout [34].

Table 1.

Contrast properties of retinal tissues. ‘+’ and ‘−’ symbols indicate high and low, respectively.

Tissue	AC	DOPU	OCTA
Inner retinal tissue	+ or −	+	+ or −
RPE	+	−	−
Choroidal stroma	+	−	+
Choroidal lumen	−	+	−

For RPE segmentation, we synthesize a new feature (F_RPE) from AC, DOPU, and OCTA as

$${\mathrm{F}}_{\text{RPE}}\equiv \text{AC}\times (1-\text{DOPU})\times (1-{\text{OCTA}}_b),$$ (1)

where OCTA_b is a binarized OCTA signal, which is created by applying Otsu’s method [35] to the raw OCTA image. AC is represented as the decadic logarithm of the attenuation coefficient in mm⁻¹ unit. Since RPE has high AC, low DOPU, and no OCTA, this feature takes on large values for RPE pixels. If F_RPE is greater than or equal to 0.15, the pixel is classified as RPE. The center-of-gravity in depth of the RPE pixels is computed for each A-line to reduce classification errors. If the depth position of a supposed RPE pixel is more than 50-pixel above (190 μm in tissue) or lower than 10 pixels (38 μm in tissue) from the center of gravity, then that pixel is considered to represent a segmentation error, and is removed.

For choroidal stromal segmentation, another feature (F_CS) is defined as

$${\mathrm{F}}_{\text{CS}}\equiv (1-\text{DOPU})\times {\text{OCTA}}_b.$$ (2)

If the choroidal stromal feature is greater than 0.1, then the pixel is classified as choroidal stroma. Similarly to the RPE segmentation, if a choroidal stromal pixel is more than 100 pixels (380 μm in tissue) away from its centroid, then that pixel is considered as an erroneously classified pixel and removed.

It should be noted that, in Eqs. (1) and (2), if two parts of the equation have large negative values, the feature (F_RPE or F_CS) then erroneously takes a large positive value, although it is expected to take a value that is smaller than the classification threshold for correct classification. However, this situation cannot occur. First, the DOPU values in both Eqs. (1) and (2) are determined with Makita’s noise correction and thus can have values that are smaller than 0.0 or larger than 1.0. Therefore, (1 – DOPU) in these equations may have both negative and positive values. Second, the AC is a decadic logarithm of the attenuation coefficient with units of mm⁻¹. Thus, if the attenuation coefficient is smaller than 1.0 mm⁻¹, or as an equivalent, if the mean free path of a photon is longer than 1.0 mm, the AC has a negative value. However, this is not likely to happen in our case. In addition, we enforce the positivity of the AC by forcibly setting it to zero if it took a negative value. Third, both OCTA_b and (1 − OCTA_b) can only have positive values because OCTA_b is binary. In summary, only one part of each equation, i.e., (1 – DOPU), can have a negative value. Therefore, a larger feature value always indicates that the pixel is more likely to be the target tissue.

3.2. Segmentation result

The segmentation method was validated by applying it to the OCT data from a normal subject. The multi-contrast images in Figs. 1(a)–(d) show scattering intensity (a), AC (the decadic logarithm of the attenuation coefficient in mm⁻¹ unit) (b), DOPU (c), and binarized OCTA (d). Hereafter, all AC images are presented in the decadic logarithm of the attenuation coefficient in mm⁻¹ unit. Figures 1(e) and 1(f) show segmented RPE and choroidal stroma, respectively. The segmented pixels are overlaid on the scattering intensity in Fig. 1(g), where red and green pixels indicate RPE and choroidal stroma, respectively. Segmentations were performed for all B-scans in the volume, and the RPE and choroidal stromal pixels were volume-rendered [Fig. 1(h)]. As shown in Fig. 1, our method produced reasonable segmentation results.

Fig. 1.

Multi-contrast images and segmentation results for a normal case. (a) Scattering intensity, (b) AC presented in the decadic logarithm of the attenuation coefficient in mm⁻1 unit, (c) DOPU, (d) binarized OCTA, (e) segmented RPE, (f) segmented choroidal stroma, (g) RPE (red) and choroidal stroma (green) overlaid on scattering intensity, and (h) volume rendering of RPE (red) and choroidal stroma (green). The scale bar indicates 0.5 mm × 0.5 mm.

The tissue classification method was also applied to pathologic cases with severe structural deformation of the retina. Figure 2 shows a case of PED, where the types of images correspond to those in Fig. 1. In the scattering intensity [Fig. 2(a)], hyper-penetration is visible (red arrow), which indirectly suggests that the RPE is absent. Despite the hyper-penetration, RPE appears to be present in the scattering intensity [Fig. 2(a)] and AC images [Fig. 2(b)], although it is severely detached. The appearance of RPE in the scattering image could be an artifact caused by the oil spill in aging Bruch’s membrane [36]. Although the apparent presence of RPE cannot be rejected as an artifact based only on the scattering information, the DOPU image [Fig. 2(c)] provides additional information to correctly reject it. In the DOPU image, the low-DOPU signal, an indicator of melanin, is absent at the top of the PED, so it can be concluded that the RPE is absent there. The RPE segmentation result [Fig. 2(e)] supports that conclusion.

Fig. 2.

Multi-contrast images and segmentation results for a PED case. (a) Scattering intensity, (b) AC, (c) DOPU, (d) binarized OCTA, (e) segmented RPE, (f) segmented choroidal stroma, (g) RPE (red) and choroidal stroma (green) overlaid on scattering intensity, and (h) volume rendering of RPE (red) and choroidal stroma (green). The scale bar indicates 0.5 mm × 0.5 mm.

The third example of the tissue classification method is an AMD case with hard exudates [Fig. 3]. The order of the sub-figures is identical to that in Figs. 1 and 2. In the scattering intensity, AC, DOPU, and OCTA images (panels (a)-(d), respectively), the hard exudates appear with high scattering, high attenuation, low-DOPU, and no OCTA signal, i.e., no flow. These properties are similar to those of RPE, so the hard exudates are erroneously segmented as RPE, as shown in Fig. 3 (red arrow). A possible way to distinguish the hard exudates from RPE is discussed later, in Section 5.

Fig. 3.

Multi-contrast images and segmentation results for a case of AMD with hard exudates. (a) Scattering intensity, (b) AC, (c) DOPU, (d) binarized OCTA, (e) segmented RPE, (f) segmented choroidal stroma, (g) RPE (red) and choroidal stroma (green) overlaid on scattering intensity, and (h) volume rendering of RPE (red) and choroidal stroma (green). The scale bar indicates 0.5 mm × 0.5 mm.

4. En face visualization of pathology

As demonstrated in panel (h) of Figs. 1 to 3, our method is easily applied to volumetric data. The method allows us to create several types of en face images, to clarify retinal pathology. In this section, we describe the method for creating four types of en face maps, and discuss their clinical utility.

4.1. En face map generation

A melano-layer thickness map is created by counting the number of pixels of segmented RPE at each A-line. Such a map primarily represents the thickness of the RPE. However, the RPE is severely deformed in some pathologic cases, and the melanin exists not only at the RPE but may also have migrated into other tissues. Therefore, we refer to this map as a “melano-layer thickness map” rather than an RPE thickness map.

An RPE elevation map is derived by calculating the depth centroid position of segmented RPE pixels at each A-line, i.e., the mean depth position of RPE. This type of map depicts RPE elevation, and deformations such as drusen.

A choroidal thickness map is created by detecting the anterior and posterior edges of the choroidal stroma. To detect the choroidal edges, we first apply a morphological closing operation [37] (with a 7-pixel × 7-pixel rectangle structuring element) to the segmented choroidal stroma. The uppermost and lowermost pixels are considered to be the anterior and posterior boundaries, respectively. As the name suggests, this map quantifies choroidal thickness.

We also created the choroidal stromal AC map, by using the segmented choroidal stroma as a mask to restrict the region of interest. The en face choroidal-stromal AC map is generated by computing the mean AC of choroidal stroma at each lateral position. Here the AC is averaged in linear representation and then converted into logarithmic space for image display.

4.2. En face visualization results

4.2.1. RPE disorders

Melano-layer thickness maps and RPE elevation maps were created for four eyes of four cases: a normal case, a geographic atrophy (GA) case, and two PED cases, as shown in Fig. 4 (first to fourth columns, respectively). The first row displays representative OCT cross-sections for these types of cases, and the second row displays composite images in which segmented RPE (red) and choroidal stroma (green) are overlaid on the OCT. The location of these cross-sections are indicated by dashed horizontal lines in the en face maps [Figs. 4(i)–(p)].

Fig. 4.

Examples of en face RPE analysis. The first row shows representative OCT cross-sections. The second row shows the same OCT cross-section, but with the segmented RPE and choroidal stroma overlaid as red and green pixels, respectively. The third and fourth rows are melano-layer thickness maps and RPE elevation maps, respectively. The first to fourth columns represent normal and GA cases, and two PED cases. The positions of the OCT cross-sections (first and second rows) are indicated by horizontal dashed lines on the en face images (third and fourth rows).

The third row [Figs. 4(i)–(l)] shows the melano-layer thickness maps where the brighter color indicates thickener melano-layer. In the normal subject [Fig. 4(i)], a relatively thick (bright) melano-layer is observed at the fovea, which agrees with a previous report of histological analysis [38]. In the GA subject [Fig. 4(j)], RPE atrophy is clearly observed as a zero-thickness (black) region. In the PED subjects [Figs. 4(k)–(l)], the RPE is missing and the melano-layer is thickened at the location of the detached RPE. The interpretation of the melano-layer thickness is discussed in Section 4.2.2.

Note that the retinal vascular patterns appear in the melano-layer thickness maps as zero-thickness regions (black). Similar artifacts can also be found in Fig. 1(h). These are generated by the OCTA projection artifact. Specifically, the retinal flow casts an OCTA shadow artifact on the RPE, which appears as pseudo-flow. Because our RPE segmentation algorithm assumes that no flow occurs in the RPE, this pseudo-flow leads to misclassification of the RPE as non-RPE tissue. Additionally, this ultimately causes the vascular pattern artifact that occurs in the melano-layer thickness map. These artifacts can be avoided through use of one of the previously demonstrated projection-resolved OCTA methods [39, 40]. However, we did not use such an algorithm for this demonstration. This is because this paper aims to demonstrate a method that uses multiple contrast information, and we therefore tried to avoid the use of other information as far as possible.

The RPE elevation maps [Fig. 4(m)–(p)] illustrate RPE deformation and elevation. The bulk curvature, which is caused by both the morphology of the eye globe and the geometrical depth-curvature of the OCT imaging field, is not corrected. Therefore, all of the cases, including the normal case, show large but slow variation of RPE depth-position, as indicated by the color changes. In the GA case, [Fig. 4(n)], the drusen appear as tiny elevation specks (arrows). In the PED subjects [Figs. 4(o) and (p)], the shapes of the RPE detachments can clearly be seen.

4.2.2. Melano-layer thickness map and near infrared fundus autofluorescence

The melano-layer thickness maps are very similar to the near infrared fundus autofluorescence (NIR-AF) images, as shown in Fig. 5 for three pathologic cases. The first to third columns show cases of serous PED, drusenoid PED, and exudative AMD, respectively. The first to fourth rows show the melano-layer thickness maps, NIR-AF images, representative OCT cross-sections, and the OCT cross-sections with segmented RPE pixels (red) overlaid. The locations of the cross-sections are indicated by the horizontal dashed lines in the corresponding melano-layer thickness maps. The NIR-AF images were recorded with a confocal scanning laser ophthalmoscope (HRA 2, Heidelberg Engineering, Germany) having an excitation wavelength of 788 nm and detected emission wavelength of > 800 nm.

Fig. 5.

Comparisons between melano-layer thickness maps [(a)-(c)] and NIR-AF images [(d)–(f)]. The third row [(g)–(j)] shows representative OCT cross-sections, and the fourth row [(k)-(n)] shows the same OCT images but with the segmented RPE overlaid as red pixels. The position of each cross-sectional image is indicated on the corresponding sub-figure (a)–(c). The columns represent serous PED, drusenoid PED, and exudative AMD cases from left to right.

Similar patterns can be seen between the thick melano-layer regions [first row of Fig. 5, (a)–(c)] and hyper-autofluorescent regions [second row of Fig. 5, (d)–(f)]. The melano-layer thickness map is based not only on the RPE layer but also any other melanin-containing tissue, including RPE cells that have migrated. Similarly, the NIR-AF signal becomes stronger if the tissue contains more melanin, for example in the cases of thickened RPE and RPE migration [36, 41]. Namely, both of the melano-layer thickness map and the NIR-AF become bright with any types of melanin containing tissues. These contrast properties of the melano-layer thickness map and NIR-AF, can explain the similar patterns in the figure.

Although they have similarities, the melano-layer thickness map has an great advantage over the NIR-AF as the segmented RPE image provides 3-D information, as demonstrated in Fig. 5(k)–(n). This 3-D information within the melano-layer thickness map can be used for further interpretation of the NIR-AF, such as localizing the source of hyper NIR-AF signal.

In the serous PED case (the first column), the maps show a thick melano-layer [Fig. 5(a), arrow] and a corresponding hyper-fluorescent pattern in the NIR-AF [Fig. 5(d), arrow]. The depth-resolved segmented RPE image [Fig. 5(k)] shows that this signal in the melano-layer thickness map originates from the thickened RPE, which is technically classified as sloughed RPE [36]. This strongly implies that the source of the hyper fluorescence is also the migrated RPE.

The drusenoid PED case (second column) also shows similar patterns in the melano-layer thickness map [Fig. 5(b)] and NIR-AF [Fig. 5(e)], as indicated by red arrows. The depth-resolved segmented RPE image [Fig. 5(l)] shows that the source of these appearances in the melano-layer thickness map and the NIR-AF are hyper-reflective foci (arrow).

The exudative AMD case (the third column) also shows similar patterns between the melano-layer thickness map [Fig. 5(c)] and the NIR-AF image [Fig. 5(f)], as indicated by the red arrows. Based on the depth-resolved segmented RPE map [Fig. 5(m) and 5(n)] and a previous demonstration of polarimetric imaging of AMD [42], this pattern is expected to originate from migrated melanin in the retina [Fig. 5(m), red arrow] and hyperreflective foci [Fig. 5(n), red arrow]. However, the red circles mark an example of disagreement between the melano-layer thickness map and the NIR-AF. In the circular region, the melano-layer thickness map shows a hyper signal that appears to be thickening of the RPE, but the NIR-AF shows hypo-AF. In the OCT cross-section, the RPE at the corresponding region shows a slightly irregular shape [Fig. 5(i), as indicated by the yellow half-bracket and shown magnified in the inset], which implies deterioration of the RPE. The depth-resolved segmented RPE image [Fig. 5(m)] reveals that some parts of the choroid in this region are misclassified as RPE, perhaps because of the choroidal melanin. This choroidal melanin and the relatively short wavelength used for the NIR-AF provide a partial explanation of the source of the discrepancy. Specifically, the NIR-AF signal originates not only from the RPE but also from the choroidal melanin. The NIR-AF signal originating from the choroidal melanin can be scattered or blocked well by the abnormal RPE. Therefore, the NIR-AF signal shows hypo-AF, while the misclassified RPE pixels in the choroid cause the hyper signal in the melano-layer thickness map.

4.2.3. Choroidal characteristics

To analyze choroidal characteristics, we created choroidal thickness maps and choroidal stromal AC maps for one eye from each of four subjects. Figures 6(a)–(d), respectively, show representative OCT B-scans from one nearly emmetropic eye (with spherical equivalent refractive error (SE) of -0.50 D), two moderately myopic eyes (SE -3.00 D and -3.75 D), and one highly myopic eye (SE -7.50). In Fig. 6(e)–(h), the segmented RPE and choroidal stroma are overlaid on the OCT B-scan as red and green pixels, respectively.

Fig. 6.

Examples of en face maps representing choroidal characteristics. The first row shows representative OCT cross-sections. The second row shows the same OCT cross-sections, but with the segmented RPE and choroidal stroma overlaid as red and green pixels, respectively. The third and fourth rows show choroidal thickness maps and choroidal stromal AC maps, respectively. The first to fourth columns represent normal to myopic cases with the spherical equivalent refractive errors of -0.50, -3.00, -3.75, -7.50 D, respectively. The position of the OCT cross-sections (first and second rows) are indicated by horizontal dashed lines on the en face images (third and fourth rows).

The choroidal thickness maps [Fig. 6(i)–(l)] demonstrate a trend for eyes with higher myopia to show a thinner choroid, which agrees with the common knowledge about choroidal thickness of normal and myopic subjects [43]. Although the right part of Fig. 6(j) and the left part of Fig. 6(k) show thin choroid, those regions could be artifacts caused by the large choroidal vessels visible in Figs. 6(f) and 6(g). However, the thinning at the right side of Fig. 6(l) is real, as shown in the corresponding cross-sectional image [Fig. 6(h)].

Unlike the choroidal thickness maps, the choroidal stromal AC maps [Fig. 6(m)–(p)] demonstrate that myopic eyes show a higher AC (i.e., a higher scattering coefficient) than normal eyes. This difference could be attributed to the following causes. In general, it is known that the myopic choroid is thinner than the normal choroid [43]. This difference may result in a more dense choroidal stroma in myopic eyes. The thinner choroid can also cause a higher melanin density. The difference in vascular volume fractions between the normal and choroidal stromata may also affect the AC. While the large choroidal vessels are not included in the segmented choroidal stroma, the fractions of the smaller vessels and capillaries would alter the AC. However, it is still open to question whether or not a higher vascular fraction would result in a higher or lower AC.

5. Discussion

5.1. Segmentation repeatability

To evaluate the segmentation repeatability, the same eye was scanned twice and both choroidal thickness maps and melano-layer thickness maps were created, as shown in Fig. 7. The first and second rows in the figure represent the first and second measurements, respectively. From left to right, the columns show representative OCT cross-sections with segmented RPE (red) and the choroidal stroma (green), the melano-layer thickness maps, and the choroidal thickness maps. The ‘×’ symbol represents the position of the fovea in each image.

Fig. 7.

Comparison of pairs of measurements of the same subject. The two rows represent the two different measurements. The first column shows OCT cross-sections overlaid with the segmented RPE (red) and the choroidal stroma (green). The second and third columns show the melano-layer thickness maps and the choroidal thickness maps, respectively. The ‘×’ symbols indicate the foveal positions, while the red circles and the arrow shown in (f) indicate the discrepancies between the two measurements.

While some lateral motions occurred, the melano-layer thickness maps show good inter-measurement agreement, i.e., good repeatability. The mean melano-layer thicknesses in the two cases were 14.3 ± 6.9 μm and 14.8 ± 7.5 μm (i.e., the mean ± the standard deviation over the measured area).

The choroidal thickness maps also show good inter-measurement agreement, other than at some of the focal points. The mean choroidal thicknesses in the two cases were 224.8 ± 57.1 μm and 224.5 ± 61.2 μm (i.e., the mean ± the standard deviation over the measured area). The focal disagreements did not originate from the segmentation method itself. For example, the horizontal line in Fig. 7(f) (indicated by the red arrow) is caused by rapid eye motion. The thin horizontal tracts [highlighted by the red circles in Fig. 7(f)] were caused by the nonoptimized depth location of the choroid. Specifically, the choroid was located too close to the zero-delay and went partially beyond the depth measurement range.

5.2. Comparison with other segmentation methods

The segmentation results were compared with the results of manual segmentation performed by an expert ophthalmologist (indicated by MM) and also with the results obtained using our previously demonstrated chorio-scleral interface (CSI) segmentation method, which was based on phase-retardation PS-OCT imaging [44], as shown in Fig. 8. The first row compares the RPE segmentations, where the segmentation results obtained by the present method and those obtained by the expert are displayed using cyan and magenta colors, respectively. The second row compares the choroidal segmentation results. The green pixels indicate the results obtained using the present method and the red curves indicate the manually segmented CSI determined by the expert. Figure 8(e) also shows a blue curve, which indicates a CSI that was automatically segmented using our previously demonstrated automatic segmentation method, which was based on phase-retardation PS-OCT [44]. Because this older algorithm is not applicable to pathologic cases, it was performed only for the normal case. The columns from left to right represent the normal case, the GA case, and two PED cases, which correspond to the cases shown in Fig. 4.

Fig. 8.

Comparisons of results obtained using the present method, manual segmentation, and the previously demonstrated PS-OCT-based chorio-scleral interface (CSI) segmentation method [44]. From left to right, the columns represent the normal case, the GA case, and two PED cases. The first row compares the results of RPE segmentations using the present method (cyan pixels) with those obtained by manual segmentation (magenta). The second row compares choroidal stromata segmented using the present method (green) and CSIs delineated using the previous PS-OCT-based method (blue) and by a human expert (red). Because the previous PS-OCT-based method is not applicable to pathologic cases, it is only shown in (e).

The figures show that the present method and the manual segmentation method gave similar RPE segmentation results with the exception of two remarkable differences. The first difference occurs at the atrophic region of the GA case [Fig. 8(b), as indicated by the arrows], where some parts of the choroid were erroneously classified as RPE by the automatic segmentation method. It seems that the absence of RPE in this region altered the scattering-image properties of the choroid. This also resulted in the vascular pattern artifact shown in the melano-layer thickness map [Fig. 4(j)] in this atrophic region. The second difference occurs at the top of the pigment epithelial detachment in the second PED case [Fig. 8(d), arrow]. Here, the automatic segmentation method depicts a thickened RPE, while the manual segmentation results delineate the RPE as a thin layer. In the corresponding OCT cross-section [see Fig. 4(d)], the RPE appears with an unclear border. Therefore, it is difficult to determine whether manual segmentation is more rational than the automatic segmentation method.

When the present method and the manual segmentation method were compared, the RPE segmentation results showed root mean square (RMS) depth-position differences of 7.7 ± 6.8 μm, 24.8 ± 20.2 μm, 14.2 ± 9.4 μm, and 13.6 ± 11.4 μm for the normal, GA, and first and second PED cases, respectively. Here, the depth-position of the RPE at each lateral position is defined using the depth centroid of the RPE pixels. In the normal case, the error is approximately twice the pixel-depth separation (4.0 μm) and is close to the resolution limit of the melano-layer thickness measurement (11 to 12 μm; see the third paragraph of Section 5.6 for details). By factoring in the pixel depth separation, the resolution limit, and the expected manual segmentation accuracy, the depth position difference for the normal eye would be rational. Previous demonstrations of morphological analysis-based automatic segmentation methods showed segmentation errors of approximately 10 to 20 μm [11]. While the differences between the datasets used makes direct comparison difficult, our nonmorphological method shows accuracy that is comparable with that of the previously demonstrated morphological information-based segmentation method.

The CSI that was segmented using the present method shows good agreement with that segmented using the old method [44] in the normal eye [Fig. 8(e)], except in the nasal region (on the right in the image). The manual segmentation results agree well with those of the old method in this region. Therefore, the present method would erroneously classify part of the sclera as a choroidal stroma. Similar misclassification of the sclera can be found in the atrophic region in the GA case [Fig. 8(f)], the nasal section (right) and the sclera beneath the apex region of the PED in the first PED case [Fig. 8(g)], and the nasal section (left) of the second PED case [Fig. 8(h)]. In these erroneous regions, we find strong signals from the sclera. Therefore, we believe that hyper-penetration can cause this type of misclassification. Because the sclera is known to have strong birefringence [25, 45, 46], further improvement of the algorithm based on additional use of birefringence information may be able to resolve this issue.

In contrast, insufficient penetration into the deep tissue can also lead to an artifact. Our choroidal thickness estimation method uses the noise-corrected DOPU and noise-corrected OCTA signals, which fade away in the deeper low-signal-to-noise-ratio regions. Therefore, if the penetration into the deep tissue is insufficient, the lower part of the choroidal stroma can be erroneously classified as nonchoroidal tissue. In addition, if these deep misclassified regions are connected to the large choroidal vessels, this can results in the incorrect appearance of thin regions in the choroidal thickness maps. This would then cause the vascular pattern artifacts shown in Fig. 6(i)–(l). This problem could be mitigated by either improving the signal-to-noise ratio using statistical methods, such as frame averaging [47] or maximum likelihood estimation [48, 49], or supplementary use of structure-based choroidal-vasculature envelope detection methods, such as that described in Section 4.2 of Ref. [50].

5.3. Computation time

Some segmentation methods that can be applied to OCTs with pathologic macular layers have been reported [9–12], but these methods employ complicated algorithms and iterative processes, and therefore are computationally intensive and time-consuming. Our algorithm, instead, is simple and does not rely on iteration. In addition, since our algorithm uses prior knowledge about tissues’ optical contrast, it does not require a classifier-training process. The computation time for this proposed method is less than 5 min for 580 × 500 × 256-pixel 3-D volume, i.e. 1.17 s/B-scan. This computation time includes not only the core segmentation process but also inessential processes such as data loading and saving. The core segmentation processing takes only around 1 min for the volume, i.e. 0.25 s/B-scan. The algorithm was implemented in Python 2.7 and processed on a PC with an Intel i7-6820HQ CPU@2.70GHz processor and 16GB of RAM.

Because the present study intends to demonstrate the concept of nonmorphological pixel-wise segmentation using JM-OCT, we ensured that the algorithm was as simple as possible. However, the algorithm can made be faster or more robust by combining it with other conventional segmentation methods. For example, by segmenting the inner limiting membrane using standard structure-based segmentation methods, we can exclude the vitreous humor before application of the present pixel-wise method. This will make the process faster and/or more robust. As we discussed in Section 5.2, our choroidal-stromal segmentation procedure is still not highly accurate. The additional use of conventional intensity-based CSI segmentation methods such as those in Refs. [51, 52] can improve the accuracy of the segmentation process.

5.4. Relation to machine learning-based segmentation

The method presented in this paper can be understood by analogy to a simplified machine-learning pixel-wise segmentation. We combined multiple optical properties into a single new value, a so-called “feature”. This process is comparable to empirical feature engineering in which only a single feature is created. In addition, the application of thresholding to the feature is regarded as a way to divide a one-dimensional feature space into two sub-domains. So, the present method has a great similarity to a feature-space based machine learning classification. There are two notable points about our method. First, the feature is manually defined, based on prior knowledge of the optical properties of the tissue. Second, the region boundary is determined empirically by thresholding with a constant threshold value.

In the current algorithm, the threshold value for segmentation is determined empirically. The analogy to the machine-learning based method implies that the threshold value could be automatically determined by one of the standard machine-learning methods, such as perceptron or support vector machines. However, this automation requires a training dataset, and the preparation of the training dataset is burdensome to human experts. Hence, it remains an open question whether the machine learning method is the best option.

5.5. Segmentation of hard exudates

As described in Section 3.2, hard exudates can be misclassified as RPE, because our RPE feature is designed with AC, DOPU and OCTA, and the optical properties for hard exudates and the RPE are similar. However, hard exudates appear with high birefringence, as shown in Fig. 9(c), so introducing birefringence to the RPE feature may enable the separation of RPE from hard exudates.

Fig. 9.

The process to distinguish hard exudates from RPE. (a) shows a representative OCT cross-section (scattering intensity). (b) shows the RPE as segmented by the method presented in Section 3.1. (c) The corresponding birefringence cross-section. (d) The feature for hard exudate segmentation [F_HX, Eq. (3)] (e) The binary map created from (d) by applying a threshold and subsequent morphological filtering. (f) Segmented RPE (red) and hard exudates (blue) overlaid on the OCT cross-section. The scale bar indicates 0.5 mm × 0.5 mm.

Figure 9 shows our preliminary attempt to distinguish the hard exudates from the RPE, in which we use the birefringence and additional morphological information to generate a new feature for hard exudates. Figure 9(a) shows an OCT cross-section with a large area of hard exudates (reproduced from Fig. 3(a)). The first step is RPE segmentation, as described in the previous section, to produce the segmented RPE Fig. 9(b), which is identical to Fig. 3(e). It is evident here that the hard exudates have been misclassified as RPE. As shown in the corresponding birefringence cross-section [Fig. 9(c)], the hard exudates possess high birefringence. Therefore, a new feature for hard exudates is defined as

$${\mathrm{F}}_{\text{HX}}(x,z)\equiv \text{BR}(x,z)\times \mathcal{N}{[D(x,z)]}^2\times {\mathrm{M}}_{\text{RPE}}(x,z),$$ (3)

where BR is the birefringence value and D is the axial distance from the inner limiting membrane (ILM) to the pixel of interest. $\mathcal{N}[D(x,y)]$ represents a normalization operation which scales a 2-D map (D(x, y)) as to make the minimum value 0 and the maximum value 1. M_RPE is a binary mask made from the segmented RPE. It equals 1 for RPE pixels and 0 for non-RPE pixels. x and z are lateral and axial positions, respectively. Although these variables were omitted in the previous equations [Eqs. (1) and (2)], they are explicitly written in this equation to show that this feature includes morphological information. The ILM was identified by selecting the topmost unity pixels in another feature

$${\mathrm{F}}_{\text{ILM}}(x,z)\equiv ℬ[\mathcal{N}[\text{OCT}]\times (1-{\text{OCTA}}_b)],$$ (4)

where $ℬ[]$ represents binarization with an empirically defined threshold of 0.4. The two-dimensional map of F_HX(x, z) is shown in Fig. 9(d). In that map, the exudates show high values because they have high birefringence and are relatively close to the ILM. Although RPE sometimes shows relatively high birefringence artifact, F_HX(x, z) becomes low because the RPE is farther from the ILM than the exudates are. The hard exudate pixels are finally segmented by application of the threshold to the hard exudation feature as F_HX(x, z) > 0.05, subsequent morphological closing with a × 9-pixel rectangle structuring element [Fig. 9(e)], and multiplication again by M_RPE(x, z) Figure 9(f) shows the final segmented result in which the RPE pixels (red) and hard exudate pixels (blue) are overlaid on the OCT cross-section.

It should be noted here that the hard exudates are also known to appear with low DOPU [24, 53]. Therefore, the high birefringence of these exudates could in fact be an artifact rather than actual high birefringence. While the present trial implementation of the process did work, further investigation will be necessary to provide more robust hard-exudate segmentation.

5.6. Interpretation of “melano-layer thickness”

In this study, we have used the term “melano-layer thickness” to denote the total thickness of all pixels that were classified as RPE, which are denoted as “RPE pixels,” at each A-line. While this terminology provides an intuitive understanding, it may also be misleading. Therefore, we clarify its interpretation here.

The melano-layer thickness is proportional to the number of RPE pixels contained in an A-scan. It should be noted here that these RPE pixels are not necessarily connected to each other. All isolated RPE pixels, such as those shown in Fig. 5(m), are included in the computation of the thickness. While the term “layer thickness” is correct in the majority of cases, it is not an accurate expression in certain cases, such as that shown in Fig. 5(m). This point should be carefully considered when interpreting the melano-layer thickness.

The depth resolution of the segmentation is another factor that must be taken carefully into consideration for the interpretation. The DOPU was used to segment the RPE pixels. The DOPU was computed using a 3 × 3-pixel kernel, which corresponds to physical dimensions of 11 to 12 μm (depth) and 35 μm (lateral) in the tissue. Therefore, the depth resolution of the DOPU is less than 11 or 12 μm, and the resolution of the melano-layer measurement is thus also less than 11 or 12 μm. This limitation of the resolution should also be considered as part of the interpretation.

6. Conclusion

We have proposed a pixel-wise segmentation method, which uses a “feature” synthesized from multi-contrast images obtained by JM-OCT. Our method is applicable to retinal tissue with severe structural abnormalities, as we have demonstrated by performing RPE and choroidal stromal segmentation in normal and pathologic cases including PED, GA, and exudative AMD. Several clinically useful en face maps were generated from the segmentation results. The melano-layer thickness map, RPE elevation map, choroidal thickness map and choroidal stromal AC map reveal the morphological and optical properties of segmented RPE and choroidal stroma. We showed that the melano-layer thickness maps are very similar to NIR-AF images, and hence can be used to identify the source of hyper-AF. We conclude that the proposed segmentation method will be useful for both qualitative observation and quantitative analysis of macular disease.

Funding

Japan Society for the Promotion of Science (JSPS, 15K13371); the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) through a contract of the Regional Innovation Ecosystem Development Program, Tomey Corporation, and TOPCON.

Disclosures

SA, AM: Tomey Corp. (F), TOPCON (F), Nidek (F), Kao (F); SM, YY; Tomey Corp. (F, P), TOPCON (F), Nidek (F), Kao (F); YI: Bayer (F), Tomey Corp. (F); MM Allergan (F), Alcon (F), Novartis (F,R), Santen (F,R), Bayer(F).