Improved biometric quantification in 3D ultrasound biomicroscopy via generative adversarial networks-based image enhancement

Abstract

This study addresses the limitations of inexpensive, high-frequency ultrasound biomicroscopy (UBM) systems in visualizing small ocular structures and anatomical landmarks, especially outside the focal area, by improving image quality and visibility of important ocular structures for clinical ophthalmology applications. We developed a generative adversarial network (GAN) method for the 3D ultrasound biomicroscopy (3D-UBM) imaging system, called Spatially variant Deconvolution GAN (SDV-GAN). We employed spatially varying deconvolution and patch blending to enhance the original UBM images. This computationally expensive iterative deconvolution process yielded paired original and enhanced images for training the SDV-GAN. SDV-GAN achieved high performance metrics, with a structural similarity index measure (SSIM) of 0.96 and a peak signal-to-noise ratio (PSNR) of 36.92 dB. Structures were more clearly seen with no noticeable artifacts in the test images. SDV-GAN deconvolution improved biometric measurements made from UBM images, giving significant differences in angle opening distance (AOD, p < 0.0001) and angle recess area (ARA, p < 0.0001) measurements before and after SDV-GAN deconvolution. With clearer identification of apex, SDV-GAN improved inter-reader agreement in ARA measurements in images before and after deconvolution (intraclass correlation coefficient, [ICC] of 0.62 and 0.73, respectively). Real-time enhancement was achieved with an inference time of ~ 40 ms/frame (25 frames/s) on a standard GPU, compared to ~ 93 ms/frame (11 frames/s) using iterative deconvolution. SDV-GAN effectively enhanced UBM images, improving visibility and assessment of important ocular structures. Its real-time processing capabilities highlight the clinical potential of GAN enhancement in facilitating accurate diagnosis and treatment planning in ophthalmology using existing scanners.

Introduction

Ultrasound biomicroscopy (UBM) is a high-frequency, high-resolution, inexpensive imaging modality that is used for imaging the anterior segment of the eye. UBM is crucial for imaging ocular structures such as the scleral spur, iridocorneal angle, and ciliary body, providing insights into glaucoma pathophysiology [1]. UBM has the unique capability of visualizing ocular structures beyond the opaque iris (i.e., ciliary body) or in case the transparency of the anterior segment is compromised due to an injury or ocular diseases. Commercially available UBM systems (i.e., Ellex EyePrime or Quantel Aviso) provide only 2D views of the anterior segment that do not provide sufficient anatomical context [2]. This limitation necessitated the development of 3D-UBM to establish the anatomical context [3–8].

Commercially available 2D-UBM systems, consisting of a mechanically swept, single transducer element, lack dynamic focusing capability, resulting in degraded image quality beyond the focal depth of the probe. This degradation occurs due to the spatially varying wide point-spread function (PSF) of the ultrasound (US) imaging system. The spatially variant PSF can cause blurring of crucial ocular structures relevant to glaucoma and other conditions. Many biometric measurements related to the anterior chamber are obtained from 2D-UBM images for glaucoma diagnosis and surgeries, including anterior chamber depth and width, lens vault, iris curvature, iris root distance, trabecular-ciliary process distance, iris-ciliary process distance, angle measurements, lens thickness, anterior lens radius of curvature, anterior segment length, and more [9–15]. Accurate measurements depend on the localization of different anatomical landmarks, which PSF blurring can impact. Deconvolution techniques mitigate the blurring effects and improve the visualization of such structures. Deconvolution approaches in imaging can be classified as blind or non-blind. In blind deconvolution, the PSF is unknown or poorly determined, and the algorithm estimates it from the image set through iterative [16] or non-iterative [17] approaches, followed by deconvolution. Non-blind or deterministic deconvolution utilizes a known PSF to enhance the image. The PSF can be measured based on the ultrasound system’s response to scatterers or predicted theoretically [18]. Previous studies have primarily focused on a spatially invariant PSF for computational simplicity [19, 20]. Although these approaches perform well within the focal depth, their performance beyond the focal plane is limited due to the severe spatial variability of the PSF away from the focal depth. Spatially variant deconvolution approaches encounter issues with patch-wise deconvolution, resulting in edge artifacts due to patch-blending problems [21]. An alternative approach involving general PSF derivation based on continually varying PSF has been proposed but is unsuitable for large images [22]. Optimization-based spatially variant PSF deconvolution methods are computationally expensive, making them challenging to implement in a real-time clinical setting. 3D-UBM generates approximately a thousand images per scan; therefore, a faster deconvolution approach is required to ensure feasibility in a clinical setting. Particularly for our 3D-UBM [3, 4], which generates approximately a thousand images per scan, a faster deconvolution approach is required to ensure feasibility in a clinical setting.

Deep learning has demonstrated remarkable success in medical imaging tasks, i.e., anatomy or disease classification, detection, and segmentation [23–28]. Generative adversarial networks (GANs), a subgroup of deep learning models, have been applied to tasks such as image synthesis, denoising, and deblurring [29–41]. GANs consist of two competing networks, a generator and a discriminator, and leverage these two networks to improve performance on a specific task. GANs can be supervised, requiring source and target domain image pairs for training, or unsupervised, eliminating the need for such pairs. Isola et al. proposed pix2pix [29], a supervised conditional GAN model based on cross-entropy and L1 loss (mean absolute error) that can perform image-to-image translation tasks. Wolterink et al. used the pix2pix framework for low-dose CT denoising [30]. Wasserstein distance has also been used along with perceptual loss and L2 loss (mean squared error) for CT image denoising [32] and deblurring [33]. Several 3D-GAN approaches have been developed in MR and CT super-resolution images [34, 37, 38]. Few studies have explored the application of deep learning in image deconvolution [39–41], especially in the case of highly spatially variant kernels such as those seen in US. Tao et al. demonstrated that using multi-scale image inputs for encoder-decoder-based recurrent network yielded the best performance in the image deblurring task on natural images [40]. Lee et al. proposed a generalized deconvolution approach to solve image degradation [41]. These studies performed blurring of natural images with a single blurring function to create blurred image-ground truth image pairs for training deep neural networks. This differs from 3D-UBM or US images, as the imaging system has a spatially variant PSF and no ground truth.

In this work, we framed the image deconvolution task as an image-to-image translation problem. We created paired training data using original images and spatially variant deconvolved images obtained via a traditional iterative approach. In a previous preliminary study [8], we evaluated the feasibility of using a supervised GAN for spatially variant deconvolution in UBM images. In this work, we expanded on our previous work by creating:

• a comprehensive analysis to assess the effects of deconvolution in visualizing small objects through a phantom and in vivo study,

• a new deep learning approach, Spatially Variant Deconvolution GAN (SDV-GAN), that demonstrates improvement in the baseline image-to-image translation model, and

• evaluation of the clinical impact of SDV-GAN on visualization and biometric measurements of ocular structures.

Materials and Methods

Image Acquisition Using 3D-UBM

We developed the 3D-UBM system using the Quantel Aviso 50 MHz probe in conjunction with a motorized translation stage (MTS50-Z8, Thorlabs Inc.) and a surgical microscope. The probe was attached to a custom-printed holder on the stage. The motor carefully moves the probe across the eye at a constant speed, enabling image acquisition in the slow scan direction (x-axis). Each image acquired is in the 2D y–z plane. The system was calibrated to acquire a total of 1000 frames in each eye. The probe was fitted with a ClearScan cover to acoustically couple the probe to the eye without introducing any discomfort. This configuration is shown in Fig. 1.

Fig. 1.

3D-UBM imaging system setup. The probe was mounted on the motorized stage with a custom 3D-printed adapter (left). Gently moving the probe across the eye at a constant speed produces a series of 2D-UBM images (middle) that can be converted to a 3D volume of the anterior segment of the eye (right)

UBM Image Enhancement via Iterative Deconvolution Approach

Figure 2 shows the processing pipeline for creating image-enhanced image training pairs for supervised training. Each 2D image (y–z) from a 3D-UBM volume was divided into N overlapping patches (50% overlap), signifying N separate depth intervals. The analytic PSF at each depth was estimated using the Field II simulator for point scatterers [42, 43]. Each patch was deconvolved iteratively using the depth-appropriate PSF. We used the Richardson-Lucy approach for a fixed number of iterations. We used five iterations, as more iterations generated more noise. Following deconvolution, the overlapped patches were blended by averaging to generate an enhanced image. We repeated the process for each 2D image in an entire 3D volume to create original and enhanced image pairs.

Fig. 2.

Enhancing UBM images using patch-wise, spatially variant deconvolution. Each image is divided into overlapping patches, and each patch is deconvolved and blended to create an enhanced image

Spatially Variant Deconvolution Generative Adversarial Network (SDV-GAN)

We approached the challenge of spatially variant deconvolution by framing it as a task of translating images from one form to another. To accomplish spatially variant deconvolution, we selected the pix2pix [29] as the backbone architecture, a variant of conditional GAN known for its success in image translation applications. Conditional GANs are particularly useful when the input is known, and the architecture of a GAN involves two networks: a generator and a discriminator. Baseline generator network is U-Net [24] shaped and was designed with an encoder, a bottleneck layer, and a decoder, connected by skip connections. The network configuration is detailed in Fig. 3. We modified each encoding layer (blue) by adding residual connections, as residual connections have shown to avoid vanishing gradient problems in deep network training by introducing shorter paths [44]. We also introduced an attention gate [45] between the decoding layer (orange) and its corresponding encoding layer to enable only relevant information to flow through the skip connections. Details of the residual block and attention gate are provided in Fig. 3. The numbers of filters in the encoder are 64, 128, 256, 512, 512, 512, and 512. Features from a decoding layer and the output of the attention gate are concatenated as inputs to the next decoding layer. The generator takes the original image $I$ as input and maps it to produce an estimate (Ĩ) of the enhanced image Ī. The discriminator employed a PatchGAN approach, distinguishing between patches from the generated enhanced image Ĩ and the real enhanced image Ī. The training process included minimizing the discriminator’s adversarial cross-entropy loss and the generator’s loss, which combined adversarial loss with the L1 loss computed between the estimated enhanced image Ĩ and the actual enhanced image Ī. Through the minimization of these losses, the generator was able to generate enhanced images.

Fig. 3.

Network architecture for SDV-GAN-based deconvolution. Original image is the input to the generator, and the generated and enhanced image pairs are inputs to the discriminator

Input images were rescaled to fall within the range of [− 1, 1], a practice employed to ensure training stability for the GAN [18]. We utilized the Adam optimizer, setting the learning rate at 0.0001 for both the discriminator and generator. A balance between L1-loss and cross-entropy loss was maintained at a 100:1 ratio. The output images generated by the generator were also rescaled within the [− 1, 1] range due to the use of the hyperbolic tangent (tanh) function as the final activation. The models underwent 100 training epochs, with the highest structural similarity index (SSIM) [31] achieved on the validation set determining the best-performing model, which was then saved for testing.

Experimental Details

Dataset

Dataset for GAN Training

We utilized a dataset consisting of 16 in vivo eye volumes obtained through the 3D-UBM system. Each image within the volume measured 1100 × 384 pixels and featured a spacing of 10 µm × 42 µm in the fast scan direction (y–z plane). Each image was cropped axially and padded laterally to 1024 × 512 for SDV-GAN input to make the input size divisible by $2^{\text{N}}$. For each eye, we captured 1000 consecutive image frames. Consequently, the spacing in the slower scan direction (x or nasal-temporal) was determined by the patient’s eye length in that dimension and the clinician’s decision. Typically, we imaged an approximately 16 mm region, yielding a spacing of 16 µm in the slow scan direction. The entire dataset was supplied by the Center for Pediatric Ophthalmology and Adult Strabismus at Rainbow Babies and Children’s Hospital under an approved institutional review board (IRB) protocol (STUDY20190764). Written informed consent was obtained from each participant. Before scanning, patients received general anesthesia. Both normal and eyes with ocular pathology were considered for GAN training. From each volume, we selected 250 images, totaling 4000 images. Out of these, 12 eye volumes (3000 images) were designated for GAN training, two volumes (500 images) for validation, and another two volumes (500 images) for testing purposes.

Dataset for Evaluation of Biometric Measurements

Forty-seven images were selected from 3D-UBM eye volumes of 18 patients with no history of glaucoma. The scleral spur was detected by three ophthalmologists (5, 6, and 18 years of experience), and the locations were averaged. Each image can be divided into two radial images, generating 94 radial images (Fig. 4). The spur could not be accurately determined in nine images, leading to 85 images for measurement. Following the accurate localization, two expert readers (a senior Ph.D. student and a senior medical student) made biometric measurements of the eye images. The measurements are trabecular-iris angle (TIA), angle opening distance (AOD), and angle recess area (ARA); all measured 500 µm away from the scleral spur [46].

Fig. 4.

Multiplanar reformatting of 3D-UBM volume and scleral spur localization. Multiplanar reformatting allows the extraction of radial images from the volume that covers the entire 360° of the anterior segment. Scleral spur can be located in these radial images, which is crucial to anterior segment biometric measurements

Performance Metrics

To evaluate the performance of deconvolution in the digital phantom and SDV-GAN in image enhancement, we employed the following metrics: (1) Contrast-to-Noise Ratio (CNR): evaluates the detectability of anatomical structures relative to the background, (2) Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE): quantify pixel-wise differences between the generated and reference images, (3) Peak Signal-to-Noise Ratio (PSNR): assesses reconstruction quality, with higher values indicating better preservation of image details, and (4) Structural Similarity Index (SSIM): measures perceptual similarity between the enhanced and reference images.

Results

Enhancing Contrast and Resolution of Small Features in UBM Through Deconvolution

Figure 5 demonstrates the effectiveness of deconvolution in enhancing image quality. In single-element mechanical sweep UBM or US imaging systems, the loss of contrast in smaller features arises due to the wide PSF at various depths. To mitigate this effect and restore the pre-convolved version of the image, deconvolution can be employed. By utilizing deconvolution, the blurring caused by the PSF can be reversed, leading to improved image quality. The simulation was carried out using the Field II ultrasound simulator. The original and enhanced “disks” were presented in log-scaled images. These disks had increasing diameters to showcase the impact of deconvolution on different-sized features. The deconvolution process significantly improved the CNR, with CNR improvement ranging from 3.9 to 10.7 dB. This improves the clarity of the disks, making them distinctly visible. We observe the highest CNR improvement in visualizing the smallest disk, demonstrating the importance of deconvolution in identifying small ocular structures. The PSF required for deconvolution was estimated based on the characteristics of the Quantel 50 MHz UBM system.

Fig. 5.

Deconvolution of ultrasound (US) images of simulated disks. During imaging, the target object is convolved with the PSF of the US imaging system (left). Deconvolution on simulated images shows CNR improvement between the original (right-top) and enhanced disks (right-bottom) on rescaled 8-bit images. The disks are of increasing diameter (25 µm, 50 µm, 75 µm, and 100 µm, respectively)

Physical Phantom Study: Deconvolution of UBM Images of a Wire Phantom

To evaluate the effectiveness of spatially variant deconvolution, we imaged a wire phantom at different depths and stitched those images together (Fig. 6). Each of the images is deconvolved using the spatially variant deconvolution approach discussed previously. Visual quality assessment shows that the point object before deconvolution suffers from more spreading away from the focus. After spatially varying deconvolution, the effects are minimized.

Fig. 6.

UBM images of a 40-µm diameter wire before (left) and after (right) deconvolution. The wire was imaged at different depths and deconvolved with depth-appropriate PSF

Effects of Deconvolution in Visualization of Ocular Anatomy

Image quality is essential for precise assessment of ocular structures in the anterior segment, especially when calculating biometrics crucial for diagnosis and treatment monitoring. Qureshi et al. observed low agreement among readers in ciliary body-related biometric measurements, attributing this to the deeper location of the ciliary body within the tissue [14]. Additionally, locating anatomical landmarks like the scleral spur and Schlemm’s canal is challenging due to the need for better contrast between different tissue types. Spatially varying deconvolution addresses the issue of image quality degradation with depth, compensating for the lack of focus outside the focal area. Figure 7 shows the effectiveness of deconvolution in visualizing small structures, i.e., Schlemm’s canal in clinical images. Schlemm’s canal is an important tissue in fluid drainage in the eye. In Schlemm’s canal visualization, contrast is greatly improved, suggesting that computational enhancement of UBM images can be useful. Figure 8 shows a 3D volume rendering of the ciliary body before and after convolution. Deconvolution improves 3D visualization in the ciliary body, which will be useful in biometric measurement.

Fig. 7.

2D-UBM image of Schlemm’s canal (red arrow) and collector channel (yellow arrow), before (red box on top-left) and after (top-right) deconvolution. Subsequent deconvolved frames also show contrast improvement (bottom)

Fig. 8.

3D volume rendering of ciliary body before and after deconvolution within the same dynamic range. After deconvolution, the edges of the ciliary body are more visible and easily distinguishable

Performance Analysis of Proposed SDV-GAN

We compared SDV-GAN performance against baseline approaches to generate enhanced images similar to the iterative deconvolution approach. SDV-GAN produces visually similar images to the iterative approach without introducing any major artifacts while also outperforming the baseline model (i.e., pix2pix). Three representative example images are shown in Fig. 9. Spatially varying deconvolution (iterative or SDV-GAN) improves the contrast of ocular structures, i.e., ciliary processes and scleral spur, in the images. Table 1 shows a quantitative comparison of the number of model parameters, inference time, and performance metrics of SDV-GAN, the baseline model without any attention gate or residual blocks, the baseline model with an attention gate, and the baseline model with residual blocks. We calculated commonly used metrics for image-to-image translation: mean absolute error (MAE), root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). For 8-bit images, the mean MAE and mean RMSE were ~ 0.97% and ~ 1.43% of the intensity range. The mean PSNR value of 36.92 dB shows excellent reconstruction quality. SSIM has been proposed as a better indicator of image similarity compared to RMSE and PSNR. The SSIM range is [− 1,1]. Our proposed approach achieves a structural similarity score of 0.96, which ensures high similarity between the iterative and SDV-GAN approaches. One-way ANOVA test on SSIM scores shows a p-value < 0.00001, indicating the mean of SSIM of SDV-GAN in test images is significantly different than the baseline, the baseline model with an attention gate, and the baseline model with residual blocks.

Fig. 9.

Comparison of SDV-GAN and iterative deconvolution images. Original, iteratively enhanced, and SDV-GAN-enhanced images are shown in the first, second, and third columns, respectively. SDV-GAN produces visually similar images to the iterative approach and shows higher contrast in distinguishing ciliary processes (top), small anechoic region (middle), and scleral spur localization (bottom) compared to images with no enhancement

Table 1.

Performance comparison of different models with respect to iterative deconvolution on 8-bit test images, best performance hihglighted in bold

Models	Metrics
	Total generator parameters (in millions)	Inference time (ms)	Structural similarity index (SSIM)	Mean absolute error	Root mean squared error	Peak signal-to-noise ratio (PSNR)
Baseline (pix2pix)	54.43	31.76	0.85 ± 0.01	4.91 ± 0.13	6.97 ± 0.83	31.32 ± 0.93
Baseline + attention gate	57.86	33.94	0.86 ± 0.02	4.46 ± 0.57	6.86 ± 1.11	31.50 ± 1.31
Baseline + residual blocks	72.59	37.42	0.92 ± 0.002	3.42 ± 0.21	4.97 ± 0.40	34.24 ± 0.70
SDV-GAN	76.01	39.92	0.96 ± 0.001	2.48 ± 0.15	3.65 ± 0.29	36.92 ± 0.69

Attention gate learns to suppress irrelevant regions of an image while focusing on important structures relevant to a specific task [45]. In our experiment, we observed a small improvement (∆SSIM = ~ 1%) over the baseline model by introducing the attention gate in the generator. We also introduced residual connections in each encoder layer, which provide better learning by mitigating the vanishing gradient problem. Residual connections in the encoder showed good improvement over the baseline only (∆SSIM = ~ 7%) in the test images. SDV-GAN combines the effects of both the attention gate and the residual connection and improvement training of the GAN. Adding both features to SDV-GAN, we demonstrated improvement in all metrics. Figure 10 shows a representative-generated enhanced image from each model. We observed SDV-GAN’s superior performance in mimicking the computationally expensive iterative spatially deconvolution approach. The artifact in difference images comes from patch-based deconvolution and stitching. We observed less artifact in SDV-GAN-generated images compared to other models.

Fig. 10.

Image enhancement performance evaluation of SDV-GAN and baseline models in generating target-enhanced images. Sample-generated enhanced images from the baseline model (without attention gate or residual blocks), baseline with attention gate, baseline with residual blocks, SDV-GAN, and the actual target enhanced image via iterative deconvolution are shown from top-left to top-right. Bottom row shows the difference images between the output of each model and the target. Lowest mean absolute difference was observed in SDV-GAN-generated images

Effects of SDV-GAN in Biometric Measurements

Not only does deconvolution affect image quality, but it can also affect biometric measurements made from UBM images. Two experts performed biometric measurements on 85 radial UBM images that were acquired by the 3D-UBM system. Table 2 shows how biometric measurements change before and after the SDV-GAN approach on three commonly measured biometrics in UBM images. We observed significant differences in mean AOD500 and ARA500 measurements for both readers. However, differences in TIA measurements were not conclusive. All three metrics rely on the localization of anatomical landmarks, i.e., the apex of the iris recess and the scleral spur, which are crucial for diagnosing glaucoma. Accurate localization is difficult depending on the types of glaucoma, and reader variability exists even in the cases where these landmarks are visible. Deconvolution improves contrast and can lead to better localization of the apex and spur, which is crucial to these biometrics. Deconvolution can improve agreement between readers in some biometric measurements. We calculated the intra-class correlation coefficient (ICC) to evaluate the agreement between the two readers. Table 3 shows the agreement between readers 1 and 2 before and after deconvolution for three separate measurements. Agreement improved after deconvolution for the ARA measurement, while agreement in TIA and AOD measurements did not change.

Table 2.

Comparison of biometric measurements before and after deconvolution

	Reader 1	Reader 2
TIA500	Significant difference in mean (p < 0.0001)	No significant difference in mean (p > 0.05)
AOD500	Significant difference in mean (p < 0.0001)	Significant difference in mean (p < 0.0001)
ARA500	Significant difference in mean (p < 0.0001)	Significant difference in mean (p < 0.0001)

Table 3.

Inter-reader agreement (ICC-score) between readers 1 and 2 before and after deconvolution

	Before SDV-GAN	After SDV-GAN
TIA500	0.91	0.90
AOD500	0.92	0.92
ARA500	0.62	0.73

Speed Comparison Between Iterative and GAN-Based Approaches

We compared the performance of deconvlucy function of MATLAB with our proposed GAN-based deconvolution. Iterative Lucy-Richardson approach has a time-complexity of $O\left(m,n\right)$, where $m$ and $n$ are the number of rows and columns in the image. Also, the algorithm scales linearly with respect to the number of iterations. 3D-UBM produces volumes of dimension 1100 × 384 × 1000. To process a single frame with a constant PSF across the image, with iteration number, i = 5, MATLAB’s implementation takes an average of 85 ms to complete. With patch-wise spatially variant deconvolution, MATLAB’s implementation takes an average of 93 ms to complete. GAN-based deep learning approaches (baseline or SDV-GAN) can process a single frame within 32–40 ms, leading to almost 3 × faster processing (Table 1). Attention gate provides a small increase in model performance without introducing many learnable parameters. In comparison, residual blocks increase model performance by a large margin, at the expense of introducing large numbers of model parameters. SDV-GAN incorporates both features. The increased number of model parameters and slightly higher inference time is justified with higher performance, and it does not impact clinical usability. All experiments were performed in an Alienware Aurora R13 PC (24-core CPU, 128 GB RAM, NVIDIA RTX 3090) and MATLAB R2022b.

Conclusions

In this work, we described a spatially variant deconvolution approach using a generative adversarial network (SDV-GAN) for improved visualization of small ocular structures. US imaging systems have depth-dependent, spatially varying PSF. Spatially variant deconvolution approaches are not trivial and do not scale well for large images. 3D-UBM and clinical UBM produce high-resolution US images of the anterior segment of the eye. From these images, several biometrics (e.g., TIA, AOD, ARA) are measured for glaucoma diagnosis and monitoring. Our experiments showed that biometric measurements before and after deconvolution show significant changes in AOD (p < 0.0001) and ARA (p < 0.0001) measurements. We also observed reader agreeability improvement (ΔICC = 0.11) after deconvolution in ARA measurement. This is likely due to the better contrast leading to better localization of the apex post-deconvolution.

SDV-GAN can mimic computationally expensive spatially variant iterative deconvolution approaches and be deployed in real-time US analysis. 3D-UBM produces 1000 images per eye volume at 10–15 frames/s. Spatially variant deconvolution in real time is a scaling challenge. To overcome this, we developed SDV-GAN, where iterative spatially variant deconvolved images were used as ground truths for training. Comparison between ground truth and generated images shows high structural similarity (SSIM = 0.96 ± 0.001), ensuring the generated images are similar to what we observe via the iterative approach. In our experiments, we found that the iterative approach can process a single UBM frame (1100 × 384) in 85–93 ms, leading to a frame rate of approximately 10–12 frames/s. In reality, it will be lower in low-performance computers that US systems are shipped with, or in point-of-care US. In comparison, our SDV-GAN-based approach processes a single image frame in 40 ms, which leads to a processing frame rate of ~ 25 frames/s. Commercially available UBM probes (Quantel, EyePrime) have frame rates in the range of 10–15 frames/s. This implies the real-time processing of images is possible, allowing clinical use. With the added complexity of SDV-GAN compared to the baseline model, it is slightly more computationally expensive. However, the real-time processing is not affected. Since our proposed approach improves biometrics measurements, even slightly slower processing speeds may be acceptable in clinical workflows where real-time visualization is not always required (e.g., post-processing for diagnostic analysis). There are a few limitations to this study. First, the dataset consists of only 16 eyes, which limits the statistical power of our findings. Additionally, there is an inherent correlation among slices from the same eye, which may affect model training and evaluation. While our approach leverages a large number of images, the dependence among slices must be considered when interpreting results. Furthermore, the current study focuses primarily on healthy eyes, and broader validation in diverse patient populations, including those with ocular pathology, is necessary. Future work will involve multi-center studies with a larger, more diverse cohort to improve the generalizability and clinical applicability of the proposed method. Second, as our training pairs were created using patch-based spatial deconvolution, enhanced images suffer from small edge artifacts. This could be solved with a better stitching algorithm, e.g., a higher overlap percentage at the cost of reduced processing speed. However, it is possible to create training pairs with more computationally intensive spatially variant deconvolution approaches and train the SDV-GAN using those images. This shows the versatility of SDV-GAN in modeling different approaches while maintaining its processing speed.

Author Contribution

All authors contributed to the study conception and design. Imaging data acquisition was performed by Faruk H. Örge and Ahmed Tahseen Minhaz. Ahmed Tahseen Minhaz and Archana Murali performed material preparation, image processing, and analysis. The first draft of the manuscript was written by Ahmed Tahseen Minhaz, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

This project was funded in part by an internal source at CWRU, the Case-Coulter Translational Research Partnership, and research funds from the state of Ohio, i.e., JobsOhio and Ohio Technology (ODSA) (TECG20200278). This was also funded in part by the Department of Defense-CDMRP Vision Research Program (W81XWH2110659).

Declarations

Ethics Approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the institutional review board (IRB) of University Hospitals (STUDY20190764).

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Consent for Publication

The authors affirm that human research participants provided informed consent for the publication of the images in Figs. 2–4 and 7–10.

Competing Interests

The authors declare no competing interests.