Super-resolution of Retinal Optical Coherence Tomography Images Using Statistical Modeling

Abstract

Background:

Optical coherence tomography (OCT) imaging has emerged as a promising diagnostic tool, especially in ophthalmology. However, speckle noise and downsampling significantly degrade the quality of OCT images and hinder the development of OCT-assisted diagnostics. In this article, we address the super-resolution (SR) problem of retinal OCT images using a statistical modeling point of view.

Methods:

In the first step, we utilized Weibull mixture model (WMM) as a comprehensive model to establish the specific features of the intensity distribution of retinal OCT data, such as asymmetry and heavy tailed. To fit the WMM to the low-resolution OCT images, expectation–maximization algorithm is used to estimate the parameters of the model. Then, to reduce the existing noise in the data, a combination of Gaussian transform and spatially constraint Gaussian mixture model is applied. Now, to super-resolve OCT images, the expected patch log-likelihood is used which is a patch-based algorithm with multivariate GMM prior assumption. It restores the high-resolution (HR) images with maximum a posteriori (MAP) estimator.

Results:

The proposed method is compared with some well-known super-resolution algorithms visually and numerically. In terms of the mean-to-standard deviation ratio (MSR) and the equivalent number of looks, our method makes a great superiority compared to the other competitors.

Conclusion:

The proposed method is simple and does not require any special preprocessing or measurements. The results illustrate that our method not only significantly suppresses the noise but also successfully reconstructs the image, leading to improved visual quality.

Introduction

Optical coherence tomography (OCT) is a noninvasive imaging technology that captures cross-sectional, two- and three-dimensional views of tissue structures; retinal OCT imaging is a clinical method for diagnosing many ocular damages.[1,2] However, there are two major challenging issues which are preventing the development of OCT-based diagnostics. First, because of the low coherence interferometry imaging mode,[3] OCT images are inevitably polluted by heavy speckle noise, which severely reduces the quality of OCT images and also the accuracy of diagnosis of ocular disorders. Noiseless OCT images are most often generated in commercial scanners by registering and averaging many B-scan OCT images captured at the same position, repeatedly. The second issue arises as a result of this recording method. It is almost impossible to capture OCT images from exactly same position for averaging, because of eye movement or unconscious jitter.[4] As a result, a low sampling rate is employed in clinics to speed up the acquisition procedure and limit the impact of unconscious movements. Hence, it is important to propose a method that operates well on both OCT image denoising and super-resolution (SR) to restore high-resolution (HR) OCT images.

Over the last two decades, several approaches have been offered to address these challenges. Methods for OCT denoising are primarily separated into hardware-based and software-based approaches.[5,6] Hardware-based methods usually work on developing the light source of the imaging system.[7,8] These methods can reduce the noise of scanners and detectors. In software-based methods, different kinds of digital filters can be applied from a global or local statistical modeling point of view in both spatial and sparse domains.[9,10,11] Besides, for the super-resolution application of OCT images, some papers are reported on sparse domain.[12,13,14]

In our previous study,[15] we proposed a method to denoise the OCT images and now we address a method for SR problem. It is generally modeled as Eq. 1

Where y is a low-resolution (LR) observed image, x is an unknown HR image to be estimated, H and D are blurring and decimation operator as 2D signals, respectively, and n represents the additive white Gaussian noise. For the SR of OCT images, H considers the identity matrix. Here, we present a SR algorithm based on statistical modeling of retinal OCT images and expected patch log-likelihood (EPLL)[16] as a patch-based super-resolution method, which has the following contributions:

• Our previous study aimed at noise reduction of retinal OCT images. Whereas, now, we intend to reconstruct the retinal OCT images with a higher number of A-scans (upsampling with different scales in lateral direction) and high resolution together

• Previous studies related to the EPLL algorithm have presented the effect of the image prior to Gaussian distribution on the different processing tasks such as noise reduction, deblurring, and inpainting for the natural images. Now, we handled this algorithm for the first time with the retinal OCT images in order to both denoising and SR applications

• Furthermore, from a modeling point of view, in contrast to previous EPLL-related works, we provide the needed presumption prior by finding the appropriate distribution and converting it to the Gaussian distribution.

In this regard, this article is organized as follows. In the next section, we describe the way of reconstructing HR-OCT images that include statistical modeling, denoising, and SR. In the result section, we present our method results and compare it with some well-known SR methods quantitatively and qualitatively. At the end, we present the conclusion of the work.

Methods

The proposed method to restore HR-OCT images can be described in some parts including preprocessing, statistical modeling, denoising, and SR.

Statistical modeling and denoising

First, the multiplicative noise of LR OCT images was converted to an additive Gaussian noise using the logarithmic operator. Then, Weibull mixture model (WMM), as an appropriate model for OCT image’s statistical characteristics such as heavy-tailed and asymmetric distribution, is fitted to the normalized intensity histogram of OCT image.[15] It is formulated as Eq. 2.

Where α_ι and β_ι are scale and shape parameters of the i^th component of the aforementioned model, respectively, and w_ι, with is nonnegative weight of each component. The critical problem in statistical modeling is to estimate the parameters correctly. For this aim, it is handled by expectation–maximization (EM) algorithm. This technique estimates the parameters in two steps, iteratively. In the expectation step, an auxiliary variable is defined to calculate the probability of belonging of pixels’ intensity to each component of mixture model. In the next step (maximization step), the value of the parameters is updated by maximum likelihood estimator.

After parameter estimation and fitting the WMM to the intensity of data, we aimed to denoise the LR OCT images. In recent years, patch-based processing has been used in many picture restoration techniques. The basic concept is to decompose the target image into overlapping patches, restore each one, and then combine the results using simple averaging. This idea has been shown to be extremely effective, yielding state-of-the-art results in denoising, inpainting, segmentation, deblurring, and other tasks. Hence, we select a patch-based algorithm considering spatial distance called spatially constraint GMM (SCGMM).[17] It clusters similar image patches in a specific window by k-nearest neighbor algorithm concerning some predefined exemplar patches. Then, a multivariate GMM is applied for each cluster with a specific mean vector and covariance matrix. Figure 1 is illustrated the clustering step.

Figure 1.

Steps of spatially constraint algorithm. The left image shows the clustering step. GMM: Gaussian mixture model, MAP: Maximum a posteriori, KNN: K-Nearest neighbor

Finally, a Wiener filter is used to denoise the patches. However, as we mentioned, the appropriate model for OCT images is a WMM while SCGMM needs a GMM distribution as a prior probability of image. Hence, the prior distribution of OCT images should be converted to the Gaussian distribution using histogram matching method. This step is called Gaussianization transform (GT) that the formulation is in the following:

The left side of the first Eq. 3 is the cumulative distribution function (CDF) of the Weibull model which is placed equally to the CDF of Gaussian distribution, and finally, in the second equation, the Gaussian random variable x_G is calculated in terms of Weibull random variables x_W, Weibull parameters α_i and β_ι, and Gaussian parameters μ_iG and σ_iG. More details can be found in the study of.[15] Now, we purpose to upscale the OCT images.

Super-resolution: Expected patch log-likelihood

EPLL is a patch-based method by Zoran and Weiss.[16] Its fundamental goal is to increase expected patch log-likelihood while remaining as close to the corrupted image as possible in a manner that depends on the prior model. Hence, due to its definition, it is formulated under the prior ρ as Eq. 4

Where x is the vectorized form of image and P_i is an operator to extract the i^th patch of x from all overlapping patches. Furthermore, log p(P_ix) is the log-likelihood of the i^th patch under the prior of p. Now, for the corruption model such as Eq. 1, the cost function which is purposed to minimize under the prior of p for finding the reconstructed image has the form as

Where D is a matrix with zero values for missing pixels. Due to the suggested prior, optimizing Eq. 5 may be very hard as direct. Therefore, half-quadratic splitting as an alternating optimization method was presented to solve Eq. 5. In “half quadratic splitting,” a set of auxiliary patches defined as a way in which each z_i is associated to the each overlapping patch p_ix. Eventually, the cost function is provided as

The patch P_ix is restricted to be equal to zⁱ, auxiliary variable, as β → ∞, then zⁱ → P_ix. The distance between the auxiliary patches and the patches of image x is therefore controlled by this parameter. The cost function is divided into a two-step inner minimization for a fixed value of β. First step: Fix zⁱ and then differentiate from Eq. 6 with respect to x and equal to zero and finally the x is calculated as Eq. 7

In the second step, the x would be fixed with updated value and next zⁱ is solved by the MAP estimation for each patch under the prior use. These two steps have been iterated 4–5 times before increasing β and repeating the whole algorithm. At each iteration, patches are extracted from the estimated image. The selection of β is crucial for the EPLL algorithm. In Zoran and Weiss’s study,[16] the values of β were selected as the coefficient of the inverse of variance such as and we tried our work with this coefficient and GMM prior. Algorithm 1 summarizes the proposed method for restoring HR-OCT images.

Algorithm 1.

The summary of the proposed method to super-resolved of retinal OCT image

Results

This section is divided into some subsections. First, the study dataset is described and then the quantitative metrics for evaluating the compared algorithms are mentioned in subsection “Quantitative evaluation metrics.” Finally, the SR results of retinal OCT images are reported both visually and numerically in subsection “Super-resolution results of retinal optical coherence tomography image.”

Dataset

The collected data included 13 3D macular OCT data that were obtained from Topcon OCT-1000 imaging device at Feiz Hospital, Isfahan, Iran. The x, y, z scale of the collected volumes is 650 × 512 × 128 voxels, 7 mm × 3.125 mm × 3.125 mm, voxel size 13.67 μm × 4.81 μm × 24.41 μm. We randomly selected 60 B-scans from the dataset and with 50% and 75% A-scans missing. Furthermore, we examined our results on the dataset with abnormality symptoms of pigment epithelial detachment which was captured from the device with previous specifications.

Quantitative evaluation metrics

The performance of the proposed method was evaluated using the mean-to-standard deviation ratio (MSR),[18] the equivalent number of looks (ENLs), the contrast-to-noise ratio (CNR),[19] the texture preservation (TP), and the edge preservation (EP)[20] which were compared to other SR methods. Furthermore, the proper regions of interest for each metric are selected. MSR is calculated by averaging the mean to standard deviation ratio in foreground sections of an image. The CNR metric is used to determine the contrast between foreground and background noise by taking into account both foreground and background regions. To evaluate the smoothness in background regions, ENL measures the mean and standard deviation of the background region. Within the corresponding region, a large ENL results in more noise smoothing. The preservation of texture and edge of an image after reconstruction is measured using TP and EP, respectively. The range of these criteria is 0–1, so the values close to zero indicate that the reconstruction method smoothed out the texture and blurred the edges. It may suggest a lack of proper filtering if the texture criterion is close to 1. In fact, there should be a compromise between image contrast and TP.

It is worth noting that the mentioned dataset is not included the reference or HR images which is the reason for not reporting the peak signal-to-noise ratio and structural similarity criteria.

Super-resolution results of retinal optical coherence tomography image

The results of retinal OCT image SR are illustrated in terms of both quality and quantity in Figures 2-4 and Tables 1 and 2. Furthermore, to investigate the performance of the proposed method, some candidate algorithms are implemented on our dataset and used for comparison including bicubic, Block Matching 3D (BM3D)[21] + bicubic, and sparsity-based simultaneous denoising and interpolation (SBSDI),[12] and Tikhonov.[22] In addition, the LInear estimator with Neighborhood patch Clustering (LINC),[17] which also is based on Gaussian assumption, is considered as a suitable comparison method. LINC in conjunction with our modeling approach (WMM-GT-LINC) produces a realistic restoration of OCT image with random data missing. However, for regularly 50% or 75% A-scans missing, the outcome is unsatisfactory due to the covariance matrix order. In order to resolve this issue, a random shift was applied to all pixels in each row of OCT image. After restoration, they were shifted to the true position and the final image was obtained.

Figure 2.

Visual comparison of super-resolution results on a sample low-resolution retinal optical coherence tomography (OCT) B-scan using the different algorithms: (a) A raw OCT image (X × 256) (with 50% data missing), (b) bicubic, (c) BM3D[21] + bicubic, (d) Tikhonov,[22] (e) two-dimensional-sparsity-based simultaneous denoising and interpolation,[12] (f) Weibull mixture model (WMM)-Gaussian transform (GT)-linear estimator with neighborhood patch clustering, (g) WMM-GT-spatially constraint Gaussian mixture model + bicubic, (h) the proposed method

Figure 4.

Visual comparison of super-resolution results on a sample of low-resolution abnormal retinal Optical coherence tomography (OCT) B-scan using the different algorithms: (a) A raw OCT image (X × 256) (with 50% data missing), (b) bicubic, (c) BM3D[21] + bicubic, (d) Tikhonov,[22] (e) sparsity-based simultaneous denoising and interpolation,[12] (f) Weibull mixture model (WMM)-Gaussian transform (GT)-linear estimator with neighborhood patch clustering, (g) WMM-GT-spatially constraint Gaussian mixture model + Bicubic, (h) the proposed method

Table 1.

Mean and confidence interval of 95% of the contrast-to-noise ratio, equivalent number of looks, texture preservation, expected patch, and mean-to-standard deviation ratio for 60 retinal optical coherence tomography images (with 50% data missing) reconstructed with bicubic, block matching three-dimensional + bicubic, Tikhonov, sparsity-based simultaneous denoising and interpolation, Weibull mixture model-Gaussian transform-linear estimator with neighborhood patch clustering, Weibull mixture model-Gaussian transform-spatially constraint Gaussian mixture model-bicubic, and proposed method (Weibull mixture model-Gaussian transform-spatially constraint Gaussian mixture model-expected patch log-likelihood)

	Original	Bicubic	BM3D + bicubic	Tikhonov	SBSDI	WMM-GT-LINC	WMM-GT-SCGMM + bicubic	Proposed method
CNR	2.4±0.16	2.6	4.91±0.52	4.28±0.3	7.94±0.64	4.28±0.31	5.56±0.59	6.22±0.73
ENL	27.09±0.66	27.85	41.36±6.96	18.4±0.85	12.06±0.94	55.37±5.33	38.28±5.64	70.72±9.38
MSR	4.92±0.12	5.06	5.01±0.39	5.11±0.23	6.32±0.55	6.09±0.29	6±0.48	6.41±0.61
TP	1±0.0	0.95±0.04	0.82±0.06	0.92±0.06	0.82±0.08	0.63±0.04	0.89±0.07	0.87±0.07
EP	1±0.0	0.82±0.01	0.79±0.01	0.8±0.01	0.77±0.01	0.85±0.01	0.81±0.01	0.83±0.01

BM3D: Block matching three-dimensional, SBSDI: Sparsity-based simultaneous denoising and interpolation, WMM: Weibull mixture model, GT: Gaussian transform, LINC: Linear estimator with neighborhood patch clustering, SCGMM: Spatially constraint Gaussian mixture model, CNR: Contrast-to-noise ratio, ENL: Equivalent number of looks, MSR: Mean-to-standard deviation ratio, TP: Texture preservation, EP: Edge preservation

Table 2.

Mean and confidence interval of 95% of the contrast-to-noise ratio, equivalent number of looks, and mean-to-standard deviation ratio for 60 retinal optical coherence tomography images (with 75% data missing) reconstructed with bicubic, block matching three-dimensional+bicubic, Tikhonov, sparsity-based simultaneous denoising and interpolation, Weibull mixture model-Gaussian transform-linear estimator with neighborhood patch clustering, Weibull mixture model-Gaussian transform-spatially constraint Gaussian mixture model-Bicubic, and proposed method (Weibull mixture model-Gaussian transform-spatially constraint Gaussian mixture model-expected patch log-likelihood)

	Original	Bicubic	BM3D + bicubic	Tikhonov	SBSDI	WMM-GT-LINC	WMM-GT-SCGMM + bicubic	Proposed method
CNR	3.19±0.23	3.51±0.26	5.73±0.78	2.69±0.18	11.19±1.33	6.81±0.61	7.9±1.34	9.6±1.9
ENL	27.45±0.48	30.33±0.9	50.79±9.63	5.9±0.18	11.45±0.78	94.66±18.37	47.82±21.24	142.05±67.65
MSR	5.13±0.15	5.45±0.19	5.26±0.37	3.25±0.09	7.57±0.76	7.52±0.57	7.06±0.96	7.78±1.21
TP	1±0.0	0.9±0.05	0.85±0.06	0.92±0.06	0.82±0.1	0.47±0.04	0.83±0.07	0.77±0.08
EP	1±0.0	0.79±0.01	0.79±0.01	0.66±0.02	0.79±0.01	0.81±0.01	0.8±0.01	0.82±0.01

BM3D: Block matching three-dimensional, SBSDI: Sparsity-based simultaneous denoising and interpolation, WMM: Weibull mixture model, GT: Gaussian transform, LINC: Linear estimator with neighborhood patch clustering, SCGMM: Spatially constraint Gaussian mixture model, CNR: Contrast-to-noise ratio, ENL: Equivalent number of looks, MSR: Mean-to-standard deviation ratio, TP: Texture preservation, EP: Edge preservation

In Figure 2, the result of OCT image SR with 50% data missing is placed and quantitative results can be shown in Table 1. According to Table 1, the proposed method makes a great improvement in ENL and MSR. Although in other metrics, it did not catch the first place, it is indeed competitive among them. It should be mentioned that in image restoration problem in addition to improving the contrast of the image, preserving the edge and texture is also very important. Hence, as can be observed, the proposed method not only produces great contrast but also preserves the texture and edge of the image well, unlike other competitive methods. Furthermore, the proposed method has been examined on 75% data missing of one sample. In Figure 3, the result of 75% data missing has been compared with other competitive methods on one OCT data sample. The numerical measurements are reported in Table 2. In addition to two criteria of MSR and ENL, our method has better performance in the edge preserving than other methods. Moreover, in Figure 4, the comparison of aforementioned algorithms has been implemented on an abnormal OCT sample with the characteristics mentioned in 3.1 section. As depicted in the Figure 4, the proposed SR method has a significant effect on noise reduction of this image compared to the other methods.

Figure 3.

Visual comparison of super-resolution results on a sample low-resolution retinal optical coherence tomography (OCT) B-scan using the different algorithms: (a) A raw OCT image (X × 128) (with 75% data missing), (b) bicubic, (c) BM3D[21] + Bicubic, (d) Tikhonov[22] (e) Sparsity-based simultaneous denoising and interpolation,[12] (f) Weibull mixture model (WMM)-Gaussian transform (GT)-linear estimator with neighborhood patch clustering, (g) WMM-GT spatially constraint Gaussian mixture model + bicubic, (h) the proposed method

Conclusion and Future Work

The data acquisition at a quick speed and high quality in OCT device is a critical challenge. In this study, we suggested an SR algorithm to restore the HR-OCT images. It is based on statistical modeling and GT. The results of SR were enhanced by using a well-designed statistical model that included key aspects of the intensity distribution of retinal OCT images, such as asymmetry and heavy tailed. The proposed method is simple and does not require any special preprocessing or measurements and its acceptable evaluation results confirm its outstanding performance. Furthermore, this research established two other reconstruction methodologies founded on the proposed statistical modeling. The first scenario involved the utilization of the study in[17] subsequent to the Gaussianization transform step. This approach is based on the multivariate GMM. Nevertheless, this approach encounters two major limitations. The first limitation revolves around the regular and patterned reconstruction of missing pixels, while the second limitation focuses on the denoising from OCT images with high noise levels. The first challenge was addressed by employing random displacement within each row of the image. To tackle the second limitation, other denoising algorithms can be utilized either before or after the implementation of LINC algorithm,[17] resulting in more effective noise reduction and improved image quality. In Figure 5, a visual comparison is depicted between two entities, denoted as “WMM-GT-SCGMM-LINC” and “WMM-GT-LINC.” Figure 5b highlights the impact of utilizing the denoising algorithm on the subsequent LINC.

Figure 5.

Visual comparison of super-resolution results on a sample of low-resolution retinal optical coherence tomography (OCT) B-scan using the different algorithms: (a) A raw OCT image (X × 256) (with 50% data missing), (b) Weibull mixture model (WMM)-Gaussian transform (GT)-linear estimator with neighborhood patch clustering (LINC), (c) WMM-GT-spatially constraint Gaussian mixture model-LINC

Furthermore, another method based on statistical modeling uses a simple interpolation algorithm to reconstruct the missing A-scans in OCT image. That is after reducing the noise of image with the algorithm, we use a simple interpolation method such as bicubic algorithm to reconstruct the missing A-scans. The numerical and visual results were also satisfactory with the WMM-GT-SCGMM+ bicubic. In the future, we want to combine the denoising and SR steps into a single module.

Financial support and sponsorship

This study was financially supported by the Vice-Chancellery for Research and Technology, Isfahan University of Medical Sciences (398999).

Conflicts of interest

There are no conflicts of interest.