Recurrent Self Fusion: Iterative Denoising for Consistent Retinal OCT Segmentation

Abstract

Optical coherence tomography (OCT) is a valuable imaging technique in ophthalmology, providing high-resolution, cross-sectional images of the retina for early detection and monitoring of various retinal and neurological diseases. However, discrepancies in retinal layer thickness measurements among different OCT devices pose challenges for data comparison and interpretation, particularly in longitudinal analyses. This work introduces the idea of a recurrent self fusion (RSF) algorithm to address this issue. Our RSF algorithm, built upon the self fusion methodology, iteratively denoises retinal OCT images. A deep learning-based retinal OCT segmentation algorithm is employed for downstream analyses. A large dataset of paired OCT scans acquired on both a Spectralis and Cirrus OCT device are used for validation. The results demonstrate that the RSF algorithm effectively reduces speckle contrast and enhances the consistency of retinal OCT segmentation.

1. Introduction

Optical coherence tomography (OCT) is a non-invasive imaging technique that utilizes low-coherence interferometry to generate high-resolution, cross-sectional images [7]. In the field of ophthalmology, OCT provides detailed visualization of the retina, facilitating the early detection and continuous monitoring of various retinal diseases, including age-related macular degeneration (AMD) [18] and diabetic macular edema (DME) [1,4]. In neurological diseases like multiple sclerosis (MS) [14,15], OCT has provided additional insights and potential biomarkers of disease, specifically the thinning of key retinal layers, such as the retinal nerve fiber layer (RNFL) and the ganglion cell and inner plexiform layer (GCIPL) [16,17]. Retinal OCT images offer precise thickness measurements of each retinal layer, enabling the identification of subtle changes over time and providing valuable guidance for treatment decisions [19].

However, discrepancies in retinal layer thickness measurements arise due to variations in image quality such as noise levels and speckle patterns produced by different OCT devices, posing challenges in the consistent comparison and interpretation of data across studies or clinical settings. Studies have demonstrated that RNFL thickness measured using Spectralis OCT (Heidelberg Engineering, Heidelberg, Germany) tends to be thicker than measurements obtained from Cirrus OCT (Carl Zeiss Meditec, Dublin, CA, USA) [3,9,12]. Similar discrepancies have been observed for other retinal layers [3] and across different pairs of OCT devices [9]. Consequently, standardization efforts are crucial in establishing harmonized measurements and minimizing disparities among OCT devices. These ensure a more reliable and consistent assessment of retinal layer thicknesses in both clinical practice and research endeavors.

In this paper, we propose the recurrent self fusion (RSF) algorithm aimed at reducing speckle contrast and improving the consistency of retinal OCT segmentation. The RSF algorithm builds upon the self fusion approach of Oguz et al. [11] and iteratively denoises retinal OCT images. In the downstream segmentation task, we use a deep learning-based retinal OCT layer segmentation algorithm [5,6]. To validate our approach, we utilize a substantial dataset comprising paired Spectralis and Cirrus OCT scans. Our findings reveal that the RSF algorithm effectively reduces speckle contrast in retinal OCT images and enhances the consistency of the resulting segmentation.

2. Method

The RSF algorithm is based on the self fusion method [11], which incorporates the concept of joint label fusion (JLF) [20], as shown in Fig. 1. The retinal OCT volume contains a set of B-scans $\{f_n^{(0)}\mid n=0,1,\dots ,N-1\}$, where the superscript 0 refers to the original B-scans and $n$ is an index over the $N$ B-scans. In the ${(t+1)}^{\text{th}}$ iteration, we register the neighboring $m^{\text{th}}$ B-scans $f_m^{(t)}$ to the $n^{\text{th}}$ B-scan $f_n^{(t)}$, by seeking the deformation field ${\varphi }_{m,n}^{(t)}$ given by:

$${\varphi }_{m,n}^{(t)}=\underset{\varphi }{\text{arg min}}{ℒ}_{\text{NCC}}(f_n^{(t)},f_m^{(t)}\circ \varphi )+ℛ(\varphi ),$$ (1)

where ${ℒ}_{\text{NCC}}$ is the normalized cross correlation (NCC) loss that penalizes low NCC values, $ℛ$ is a regularization term on $\varphi$ that penalizes discontinuities in $\varphi$, and ${\varphi }_{m,n}^{(t)}$ is the identity if $m=n$. To solve Eq. 1, we use the greedy reg package¹ [21]. An affine transformation is performed first, followed by a deformable registration. A window size of 5 × 5 is chosen for the NCC calculation, and the default regularization parameters are used.

Fig. 1.

Diagram of the RSF algorithm for iterative denoising. The operations $\circ$, ×, and + represent composition, pixel-wise multiplication and pixel-wise summation, respectively.

After registration, we perform a weighted summation to obtain updated denoised images. For the weights, we utilize the concept of JLF, to compute:

$${(M_n^{(t)})}_{ij}(x,y)\sum _{p=x-\gamma }^{x+\gamma }\sum _{q=y-\gamma }^{x+y}{d}_{n+i,n}^{(t)}(p,q)\cdot d_{n+j,n}^{(t)}(p,q)+{\alpha \delta }_{ij},$$ (2)

where $M_n^{(t)}$ is a matrix with entries $i$, $j\in \{-ϵ,\dots ,ϵ\}$ at the $t^{\text{th}}$ iteration, ($x,y$) is a pixel location, $\gamma$ defines a local patch size, ${\delta }_{ij}=1$ when $i=j$ or otherwise ${\delta }_{ij}=0$, $\alpha$ is a regularization term which controls the weight similarity, and $d$ is an intensity distance measure expressed as:

$$d_{m,n}^{(t)}(x,y)={\mid f_m^{(t)}\circ {\varphi }_{m,n}^{(t)}(x,y)-f_n^{(t)}(x,y)\mid }^{\beta },$$ (3)

where $\beta$ is a parameter that controls the measure, and $d_{m,n}^{(t)}(x,y)=0$ if $m=n$. The weight parameter is then calculated as:

$$w_{n,i}^{(t)}(x,y)=\frac{\sum _{j=-ϵ}^{ϵ}{(M_n^{(t)})}_{ij}^{-1}(x,y)}{\sum _{i=-ϵ}^{ϵ}\sum _{j=-ϵ}^{ϵ}{(M_n^{(t)})}_{ij}^{-1}(x,y)},$$ (4)

where $w_{n,i}^{(t)}$ is the weight for the warped B-scan $f_{n+i}^{(t)}\circ {\varphi }_{n+i,n}^{(t)}$. The updated $n^{\text{th}}$ B-scan $f_n^{(t+1)}$ is obtained through the weighted summation:

$$f_n^{(t+1)}(x,y)=\sum _{i=-ϵ}^{ϵ}{w}_{n,i}^{(t)}(x,y)\cdot f_{n+i}^{(t)}\circ {\varphi }_{n+i,n}^{(t)}(x,y),$$ (5)

where $2ϵ+1$ is the number of wrapped B-scans used in weighted summation. In this paper, we set $\alpha =30$, $\beta =2$, $\gamma =2$, $ϵ=1$, and we explore up to a total of $T=10$ iterations.

3. Results

Dataset.

Our dataset consists of 59 MS participants that were scanned contemporaneously on both a Spectralis and Cirrus OCT device with institutional review board (IRB) approval. For all 59 participants and the two OCT devices, both the left and right eye were imaged. Thus our evaluations will be on the 118 eyes, comparing Spectralis to Cirrus. Spectralis scans comprise 49 B-scans with dimensions of 496×1024 and Cirrus scans consist of 128 B-scans with dimensions of 1024 × 512. Both sets of scans cover an approximate field of view of 6 mm × 6 mm around the central fovea. The Spectralis and Cirrus scans have axial resolutions of 3.87 $\mu \mathrm{m}$ and 1.96 $\mu \mathrm{m}$, respectively.

Speckle Contrast.

By applying the RSF algorithm, we observe a continuous reduction in speckle patterns of both OCT devices as iterations progress as shown in Fig. 2(a). To assess the quantitative reduction in speckle patterns at each iteration, we evaluate the speckle contrast, $K_l$, in each retinal layer and across the entire retina in each B-scan:

$$K_l=\frac{{\sigma }_l}{{\mu }_l},$$ (6)

where ${\mu }_l$ and ${\sigma }_l$ are the mean and standard deviation of intensities in the $l^{\text{th}}$ layer, respectively, and the segmentation of each retinal layer is explained in the next paragraph. The speckle contrast for both OCT devices over 10 iterations is shown in Fig. 2(b). We observe that the speckle contrast for all layers decreases initially, before stabilizing for both devices. From Eq. 6, there is an inverse relationship between speckle contrast and the signal to noise ratio (SNR). Therefore, as the iterations progress, the SNR in each retinal layer increases. Furthermore, we notice that the decreasing rate of the speckle contrast is much higher for Cirrus scans than Spectralis scans. This is expected since Spectralis scans possess better initial image quality. It is interesting to note that while the original Spectralis and Cirrus scans exhibit different speckle contrast, their speckle contrast converges to similar values for the GCIPL, INL, OPL, ONL and OS after denoising; see Fig. 2 for the layer names.

Fig. 2.

(a) Original and denoised retinal OCT images after one, five and ten iterations on paired Spectralis and Cirrus scans for a similar cross section. (b) Speckle contrast across the 118 Spectralis scans (red) and the 118 Cirrus scans (green) over ten iterations. The value at iteration ‘0’ is the speckle contrast of the original images. Key: RNFL: retinal nerve fiber layer; GCIPL: ganglion cell layer and inner plexiform layer; INL: inner nuclear layer; OPL: outer plexiform layer; ONL: outer nuclear layer; IS: inner segment; OS: outer segment; RPE: retinal pigment epithelium complex.

Segmentation Convergence.

To investigate the impact of the RSF algorithm on downstream segmentation, we use a deep learning-based retinal OCT segmentation algorithm, to generate smooth and continuous surfaces that accurately represent the retinal layers with the correct topology [5,6]. This network was trained with 394 Spectralis and 321 Cirrus retinal OCT volumes. The ground truth for layer segmentation in these OCT volumes was obtained using AURA, a well-established retinal OCT segmentation software [8]. Manual corrections were made to the initial segmentation results obtained from AURA to ensure precise and accurate delineation of the retinal layers. It is important to note that none of the training data for the deep learning-based OCT segmentation algorithm were included in the paired scans used in the validation of the RSF method. We apply the deep learning-based retinal OCT segmentation algorithm to the paired Spectralis and Cirrus scans. We calculate both the unsigned (blue) and signed (yellow) differences between the segmentation results of consecutive iterations, as shown in Fig. 3, for both Spectralis and Cirrus scans. This allows us to evaluate the magnitude and direction of changes in the segmentation outcomes throughout the RSF iterations. These plots demonstrate a convergence in the segmentation results for the majority of retinal layers.

Fig. 3.

(a) Boxplots of the unsigned (blue) and signed (yellow) differences between the segmentation results of consecutive iterations across the 118 Spectralis scans. (b) Boxplots of the unsigned (blue) and signed (yellow) differences between the segmentation results of consecutive iterations across the 118 Cirrus scans. Key: ILM: internal limiting membrane; ELM: external limiting membrane; BM: Bruch’s membrane; see Fig. 2 for the other layer names.

Average Thickness Comparison.

After analysing the convergence of the independent segmentation results, we proceed to analyse the paired segmentation results between Spectralis and Cirrus scans. However, directly comparing their segmentation results requires OCT image registration [13] between the two scan types as the images are not aligned, see Fig. 2(a) for example. OCT image registration can be challenging due to the interpolation issues arising from the sparseness of OCT B-scans. To circumvent the registration problem, we calculate the average thickness within a circular area centered at the fovea, with a diameter of 5 mm, for both Spectralis and Cirrus scans. Then, we compare the difference in the averaged thickness for each retinal layer by subtracting the Spectralis thickness from the Cirrus thickness. The overall results for the average thickness difference between the paired scans at different iteration steps are shown in Fig. 4. We observe that in comparison to Spectralis thickness measurements, the initial thickness measurements obtained from Cirrus scans tend to be smaller in RNFL, GCIPL, OPL, OS, RPE and the overall retina, but tend to be larger in INL, ONL and IS. After applying the RSF algorithm, the thickness measurements for all retinal layers from the paired scans converge to a smaller value. Moreover, it is noteworthy that the RSF algorithm applied to Spectralis scans does not noticeably reduce the thickness difference, as opposed to its impact on Cirrus scans. This outcome is expected due to the inferior initial image quality of Cirrus scans, thus benefit more from the RSF algorithm.

Fig. 4.

Difference of the averaged thickness for each retinal layer by subtracting the Spectralis thickness from the Cirrus thickness at different denosing iteration steps for each device. The color and the size of each circle represent the mean and the standard deviation, respectively, of the thickness difference between the paired OCT scans across 118 paired scans. The value of the mean is shown in the color bar. The value of the standard deviation is proportional to the circle size, with the unit circle representing a standard deviation of 3 $\mu \mathrm{m}$.

Thickness Distribution Comparison.

To gain a more comprehensive understanding, it is valuable to analyse the differences in thickness distributions, as they capture the overall statistics of retinal layer thickness. To quantify the dissimilarity between the Spectralis and Cirrus thickness distributions, we employ the Jensen-Shannon Distance (JSD):

$$JSD(p_1\mid \mid p_2)=\sqrt{\frac{D(p_1\mid \mid q)+D(p_2\mid \mid q)}{2}},$$ (7)

where $p_1$ and $p_2$ are two probability density functions, $D$ is the Kullback-Leibler divergence, and $q=\frac{1}{2}(p_1+p_2)$. The JSD between the paired thickness distributions at different iteration steps are shown in Fig. 5. These results align with the findings from Fig. 4. Specifically, we observe a reduction in the JSD between the thickness distributions of each retinal layer as the denoising iterations progress. Furthermore, it is evident that the RSF algorithm has a more pronounced impact on the Cirrus scans than the Spectralis scans.

Fig. 5.

JSD of the thickness distributions for each retinal layer between the Spectralis and the Cirrus OCT scans at different denosing iteration steps for each device. The color and the size of each circle represent the mean and the standard deviation, respectively, of the JSD between the paired OCT scans across the 118 paired scans. The value of the mean is shown in the color bar. The value of the standard deviation is proportional to the circle size, with the unit circle representing a standard deviation of 0.2.

4. Discussion and Conclusions

Spectralis and Cirrus OCT devices are widely used in ophthalmology, but they differ in terms of their hardware and imaging algorithms, which impact their image quality. Spectralis scans tend to exhibit smoother images with reduced noise and fewer speckle patterns. These disparities in image quality can contribute to domain discrepancies between the two OCT scans and potentially lead to variations in retinal layer thickness measurements following segmentation. Our hypothesis is that by applying the RSF algorithm, the two distinct OCT scans gradually become more similar, resulting in improved consistency in retinal layer thickness measurements. This hypothesis is supported by the results obtained in this study.

However, the proposed method has certain limitations. First, the processing speed of the current RSF is relatively slow due to its reliance on a classical registration method, which requires a longer processing time. Second, the validation of the proposed method is limited to a single deep learning-based retinal OCT segmentation algorithm and lacks comparison with other denoising algorithms. Third, the proposed method is evaluated on a cross-sectional study and its impact on longitudinal OCT data remains unexplored. To address these limitations, future work involves exploring the implementation of deep learning-based registration methods such as Voxelmorph [2] or Coordinate Translator [10] to improve processing speed, validating the consistency of thickness measurements after denoising using a broader range of OCT segmentation and denoising algorithms, and investigating the RSF algorithm in longitudinal OCT data for identifying subtle retinal layer thickness changes over time.

In this paper, we propose the RSF algorithm for iteratively denoising retinal OCT images. Each RSF iteration effectively reduces speckle contrast and improves SNR across various retinal OCT layers and on both investigated devices. Moreover, by applying a deep learning-based retinal OCT segmentation algorithm to the paired OCT volumes from different OCT devices, we observe a significant improvement in the consistency of the segmentation results. These findings underscore the potential of the proposed RSF algorithm as a valuable pre-processing step for retinal OCT images, facilitating more consistent and reliable retinal OCT segmentation.