Skip to main content
Journal of Biomedical Optics logoLink to Journal of Biomedical Optics
. 2024 Apr 4;29(4):046001. doi: 10.1117/1.JBO.29.4.046001

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Xiaohui Zhang a, Mimi Tan b, Mansour Nabil b, Richa Shukla b, Shaleen Vasavada b, Sharmila Anandasabapathy b,c, Mark A Anastasio a, Elena Petrova b,c,*
PMCID: PMC10993061  PMID: 38585417

Abstract.

Significance

Endoscopic screening for esophageal cancer (EC) may enable early cancer diagnosis and treatment. While optical microendoscopic technology has shown promise in improving specificity, the limited field of view (<1  mm) significantly reduces the ability to survey large areas efficiently in EC screening.

Aim

To improve the efficiency of endoscopic screening, we propose a novel concept of end-expandable endoscopic optical fiber probe for larger field of visualization and for the first time evaluate a deep-learning-based image super-resolution (DL-SR) method to overcome the issue of limited sampling capability.

Approach

To demonstrate feasibility of the end-expandable optical fiber probe, DL-SR was applied on simulated low-resolution microendoscopic images to generate super-resolved (SR) ones. Varying the degradation model of image data acquisition, we identified the optimal parameters for optical fiber probe prototyping. The proposed screening method was validated with a human pathology reading study.

Results

For various degradation parameters considered, the DL-SR method demonstrated different levels of improvement of traditional measures of image quality. The endoscopists’ interpretations of the SR images were comparable to those performed on the high-resolution ones.

Conclusions

This work suggests avenues for development of DL-SR-enabled sparse image reconstruction to improve high-yield EC screening and similar clinical applications.

Keywords: end-expandable optical fiber probe, deep-learning-based super-resolution, degradation model, Barrett’s esophagus, esophageal cancer, microendoscopy, endomicroscopy

1. Introduction

With an incidence of about 600,000 new cases and more than 500,000 deaths annually worldwide, esophageal cancer (EC) is one of the deadliest cancers worldwide. The two main histological subtypes of EC include: esophageal adenocarcinoma1 and esophageal squamous cell carcinoma (ESCC).2 Screening and early detection of ESCC are critically important to help reduce the incidence and mortality associated with EC. However, challenges posed by the extensive surface area of the esophagus, the limited field-of-view (FOV) offered by conventional microendoscopic screening probes (100 to 1000  μm),35 and the time constraints of endoscopic procedures (6 to 8 min per patient) necessitate the development of an innovative approach to enable imaging of larger fields and improve diagnostic yield.

In recent years, in-vivo optical microscopy techniques, such as confocal laser endomicroscopy (CLE) and high-resolution microendoscopy (HRME), have been applied to visualize the nuclear morphology of the esophageal epithelium and assist in differentiating neoplasia from benign tissue.3,57 In contrast to conventional tissue biopsies and histopathologic analysis, these optical techniques are non-invasive and able to provide real-time results. However, microendoscopy is currently limited by its small FOV (the average diameter of a probe is 0.3 mm) that hinders it from imaging larger tissue areas. To circumvent this limitation, we propose a novel approach using an end-expandable optical probe to increase the FOV using sparse-data imaging.8,9 By use of a sleeve mechanism that expands the end of the fiber bundle with unfused microfibers, a larger tissue FOV can be achieved for real-time diagnosis (Fig. 1). However, one major challenge for successful application of such an optical probe is the development of image processing procedures that act on the acquired sparse image data and allow formation of a microendoscopic image with enhanced quality.

Fig. 1.

Fig. 1

Conceptual 3D rendering of a settled optical fiber probe (a) and an end-expanded optical fiber probe (b), respectively. Here, the diameter of a single fiber strand can vary from 4 to 6  μm.

Deep-learning-based approaches have been developed for improving the quality of microendoscopy imaging. Ravi et al. proposed a synthetic data generation approach to train exemplar-based deep neural networks for recovering HR probe-based confocal laser endomicroscopy images.10 To address the need for non-reference image quality (IQ) improvement, Szczotka et al. designed a zero-shot super-resolution approach with a physically-motivated downsampling Voronoi kernel tailored to the acquisition physics by incorporating fiber bundle geometry and noise simulation.11 More recently, a multi-frame super-resolution algorithm exploiting bundle rotation to extract features and reconstruct underlying tissue is developed by Eadie et al.12 Motivated by the advances of image super-resolution using deep learning approaches,8,1016 here we proposed a deep-learning-based image super-resolution (DL-SR) method that estimates a HRME image from its acquired low-resolution (LR) counterpart. In this way, sparse images can be acquired by the end-expandable optical fiber probe to increase the FOV at cost of spatial resolution, which could be subsequently restored by use of a DL-SR method to enhance IQ for diagnostic purposes.

The goal of our study is to demonstrate the feasibility of the concept of the end-expandable optical fiber probe using the proposed sparse imaging methodology to achieve a wider FOV with IQ comparable to that achieved by a conventional probe of fused fiber. As a surrogate of clinical trials, computer simulation studies, also known as virtual imaging trials, provide us with an economical and convenient route to explore imaging system designs.17 Here, estimates of LR microendoscopic images of esophageal mucosa that would be acquired by use of the novel optical fiber probe are computationally simulated. This will be accomplished by degrading HRME images by use of a degradation model that incorporates the physical factors of the optical fiber probe. Subsequently a DL-SR model was employed to generate super-resolved (SR) images from the LR ones. The impact of various degradation parameters of the probe on DL-SR performance was investigated to identify optimal parameters for prototyping. Additionally, a clinically-relevant detection task of esophageal neoplasia was conducted by endoscopists to study the impact of DL-SR on task performance. The results will provide valuable guidance for future prototyping and the advancement of this novel imaging technique.

2. Methods

To demonstrate the applicability of the proposed end-expandable optical fiber probe model, DL-SR models were trained in an end-to-end manner to learn a mapping between the LR images simulated with degradation models that incorporate different optical probe parameters and the HRME images. Traditional IQ metrics were computed on simulated LR and SR images to assess the perceptual IQ and identify the optimal parameters for the probe prototyping. In addition, a human pathology reading study was carried out to evaluate the utility of the images enhanced by the DL-SR model within a screening context.

2.1. HRME Image Acquisition and Pathology Interpretation

A HRME image dataset acquired by use of a low-cost, point-of-care HRME device that imaged the esophageal epithelium of patients undergoing endoscopy for ESCC screening18 was used in the study. The HRME images were obtained sequentially from patients enrolled in a clinical trial comparing standard of care Lugol’s chromoendoscopy (LCE) to LCE+HRME at three sites: First Hospital of Jilin University (Changchun, China), the Cancer Institute at The Chinese Academy of Medical Sciences (Beijing, China), and Baylor College of Medicine (Houston, Texas, United States) from December 2014 to November 2016, approved by the Institutional Review Board at Baylor College of Medicine [IRB# H-34973]. The details of image acquisition have been described in previous studies.18,19

The HRME imaging system consisted of a compact epi-fluorescence fiber optic microscope, which provides 1000× magnification views of epithelial tissue and subcellular features to distinguish cancerous from benign tissue after staining with a topical fluorescent dye, 0.01% proflavine hemisulfate. It has been evaluated in various anatomical sites, such as the cervix, anus, mouth, throat, and esophagus.5,19 Due to its low cost and reusability, HRME can have a high impact in resource-limited global settings. The HRME principles and schematics have been described previously.18 Typically, the HRME device conducts imaging through a fused optical fiber bundle with a cross-sectional diameter of 800 and 4.4  μm lateral spatial resolution (an individual fiber strand diameter as little as 4  μm). For a conventional microscopic probe, the input and the output at the distal and proximal ends of the fibers demonstrate the same pattern. The fiber strands are positioned as close to each other as possible, and the size of the fiber strands usually defined image resolution for a registering camera of high-resolution. The light illumination measures are collected from a continuous part of tissue and registered on a camera correspondingly. Figure 2(a) demonstrates the optical fiber pattern generated by the fiber strands of the fused optical fiber bundle.

Fig. 2.

Fig. 2

Schematics of image acquisition by means of (a) a conventional fused endoscopic fiber probe; (b) an end-expandable, unfused endoscopic fiber probe.

The density of the nuclei represented the metric for diagnosis and differentiation between neoplastic and non-neoplastic images. All image sites were biopsied, and biopsy histopathology results served as the gold standard. High-grade dysplasia and ESCC were classified as neoplastic; normal squamous epithelium, esophagitis, and low-grade dysplasia were classified as non-neoplastic.

All HRME images were standardized in size of 960  pixels×1280  pixels. A Gaussian filter with standard deviation of 2 pixels was applied to remove the comb pattern introduced by the sparse data of optical fiber bundles. A contrast-limited adaptive histogram equalization with a clip limit of 0.005 was employed to enhance the image contrast.

2.2. Quality Control and Image Selection

In addition to IQ control conducted as described in the previous study,19 bioengineers further categorize the acquired HRME images into three types of perceptual image qualities: good, intermediate, and poor. Images with good quality should demonstrate an FOV where nuclei can be clearly visualized. Images with intermediate quality showed mild motion blur or defocus aberration that could affects approximately a quarter of the imaged area. Images with poor quality were classified by factors, such as severe motion blur, obstructed vision, or image corruption for half of the image area or more. Only images of good and intermediate perceptual quality were selected and used in this study.

2.3. Simulated Image Data for End-Expandable Optical Fiber Probe

A virtual imaging trial was performed to simulate LR images that would be acquired using the proposed novel optical fiber probe by use of the acquired HRME images. As shown in Fig. 2(b), one can find a schematic of the concept to an end-expandable optical fiber probe. Unlike conventional image acquisition through an optical fiber bundle [Fig. 2(a)], the end-expandable optical fiber bundle can form a “brush” at the input end [Fig. 2(b)]. The input elements of such lightguide are fiber strands, which are placed at a distance from each other and all together they represent a sparse data set. Thus, the optical fiber “brush” collects fluorescent emission from a surface partially and discontinuously. To create LR images, we processed the simulated sparse image data acquired from the end-expandable endoscopic optical fiber probe [Fig. 2(b)] by filling in non-populated pixels of the sparse images [Fig. 3(b)]. Diagrams that depict the conventional and sparse image data acquisition and processing are shown in Fig. 3.

Fig. 3.

Fig. 3

(a) 1D-schematic of the conventional image data acquisition leading to the HR images; (b) the sparse image data acquisition and reconstitution from the compressed data set into the LR images. Red elements represent a fiber strand collecting light illuminated by the tissue surface and green elements are a set of pixels displayed as average value of light intensity in the adjacent fiber. (c) 2D-illustration of the simulated sparse image data; s and m are the length of side of FOV and ROI, respectively; dx and dy are the offsets in x and y directions, respectively.

To emulate the LR images that would be acquired from the novel end-expandable optical fiber probe, a degradation model that incorporates the fiber knock-out downsampling was applied to the HRME images. Assuming a single fiber strand with a diameter of 4  μm, we considered a pixel size of 2  μm in this study. As shown in Fig. 3, given a fiber strand represented by a range-of-interest (ROI) block of m  pixels×m  pixels that resides in the center of an FOV block of s  pixels×s  pixels, a random deformation offset with maximum of dx pixels and dy pixels in horizontal and vertical direction was considered, respectively. This was designed to depict the off-center deviation when defining the ROI block. The degradation model calculated the mean value over pixels within the ROI block and filled the pixels in FOV block with the derived value. In this study, three different parameters of the degradation model were investigated to evaluate the DL-SR performance: the fiber diameter m, the inter-fiber distance s, and the deformation offset d. The schematic of the proposed degradation model is shown in Fig. 4.

Fig. 4.

Fig. 4

Illustration of degradation models that incorporated various optical fiber probe parameters including (a) offset, (b) inter-fiber distance, and (c) fiber diameter.

Figure 5 shows examples of an HRME image, an intermediate sparse image and the LR image simulated by use of the degradation model, respectively. Here, the fiber diameter m=6  μm, the inter-fiber distance s=12  μm, and the deformation offset d=1.

Fig. 5.

Fig. 5

Example of an original HRME image, the same imaging area as sparse data captured by the proposed end-expandable optical fiber probe, and the LR image restored from the sparse data.

2.4. Deep-Learning-Based Image Super-Resolution

Given an LR image ILR virtually acquired with the end-expandable optical probe, image super-resolution methods seek to produce an SR image ISR as an estimate of a HRME image that would be obtained from a conventional fused endoscopic fiber probe. However, this is a challenging ill-posed inverse problem. In recent years, DL-SR methods have been widely applied in various applications.1315,2024 Here, the well-studied super-resolution convolutional neural network (SRCNN)13 was employed to investigate the feasibility of DL-SR methods to improve quality of simulated LR images from the novel optical probe. Such analysis can be readily repeated with other more recent DL-SR approaches.14,25 The SRCNN seeks to establish a mapping from the space of simulated LR images to the space of HR images

ISR=F(Θ,ILR),

where F is the network parameterized by Θ. The SRCNN was trained by minimizing the mean squared error between generated SR images and original HRME images. The architecture of an SRCNN is shown in Fig. 6, consisting of three feedforward convolutional layers interspersed with leaky rectified linear unit nonlinearities. The filter sizes of the three convolutional layers were 9×9, 1×1, and 5×5 and the corresponding number of filters were 64, 32, and 1, respectively.

Fig. 6.

Fig. 6

Schematic of the SRCNN architecture used in the study.

The training and validation data for the SRCNN consisted of 206 and 50 paired HRME and corresponding simulated LR images, respectively. During training and validation, each image was randomly cropped into 10 patches with a size of 512×512. In the testing stage, the full-size simulated LR images were used. The SRCNN was trained with Adam optimizer26 with a learning rate of 0.0001. The network was trained for 300 epochs with a batch size of 8, and the model with best validation performance was used for downstream task evaluation. For various degradation parameters mentioned in Sec. 2.3, the SRCNN was retrained and evaluated. SRCNN were implemented with TensorFlow 2.0 and trained on NVIDIA GPUs.

2.5. Image Quality Assessment and Statistical Analysis

To assess the DL-SR performance with consideration of various degradation parameters used to simulate the LR images, traditional IQ metrics, such as peak-signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) were computed on the simulated LR and SR images. In addition, a binary detection task to determine whether esophageal neoplasia is present or not was performed by endoscopists on both SR images obtained from SRCNN and the original HRME ones. The accuracy and confidence level were evaluated to assess the task-based performance of the employed DL-SR method.

A total of four endoscopists (three experts, one novice) underwent standardized training in HRME image interpretation. Expert endoscopists were defined as having previously performed >50 HRME cases, whereas novices were new to the technology. All endoscopists viewed a set of training slides that demonstrated the features of neoplastic and non-neoplastic classification of HRME images, including nuclear size, crowding, and pleomorphism. All endoscopists were asked to interpret a series of original HRME images and generated SR images as neoplastic (high-grade dysplasia, ESCC) or non-neoplastic (normal squamous, esophagitis, low-grade dysplasia) along with their confidence level in their interpretation (high or low).

Assuming 80% power, α=0.05, and two-sided test to establish equivalence between HR and SR images with equivalence limit of 0.15, a sample size of 120 images was calculated using sample-based variance estimates without continuity correction for binary data. The sensitivity, specificity, and accuracy of HRME and SR image interpretation of individual endoscopists was compared with histopathology as the gold standard. Unpaired t-test was computed to assess for differences in the sensitivity, specificity, and accuracy in endoscopists’ interpretation of HR and generated SR images.

3. Results

3.1. Performance Evaluation of DL-SR to Super-Resolve Simulated LR Images of End-Expandable Optical Fiber Probe

The DL-SR model was first trained and tested on HRME images and corresponding LR images simulated by use of a degradation model with fiber diameter m=4  μm, inter-fiber distance s=8  μm and no offset imposed. Figure 7 shows two examples of the test images where perceptual IQ of the simulated LR images appeared to be improved by DL-SR. Figure 8 shows a zoomed-in, cutoff image patch of the original HR, simulated LR, and generated SR images and the corresponding cross-sectional profiles of optical intensity. Note that, as expected, the line profiles of the SR image aligned closer to the HR one, compared to the line profiles of the simulated LR image. Visual inspection confirms that details of subcellular features could be restored by use of SRCNN to obtain improved perceptual IQ.

Fig. 7.

Fig. 7

Examples of original HRME image, simulated LR image and corresponding SR image (top row) generated by the DL-SR method with magnification (bottom row).

Fig. 8.

Fig. 8

Cross-sectional line profile on selected images allows comparison of profiles of optical intensity associated with stained nuclei on original HR images, simulated LR images, and reconstructed SR images.

3.2. Impact of Offset, Inter-Fiber Distances and Fiber Diameter on Traditional Image Quality Metrics

In this study, the impact of different parameters considered in the degradation model on traditional IQ metrics was evaluated. The offset d was varied from 0 to 10  μm, and the inter-fiber distance s was changed ranging between 4 and 24  μm. A fiber strand diameter m ranging from 4 to 12  μm was applied. All SRCNN models were trained on paired of HRME and LR images simulated with various degradation parameter values. Traditional measures of IQ were assessed by computing PSNR and SSIM values on a test set consisting of 300 images, and these quantities are plotted on Fig. 9. In most cases, the SR images generated by the SRCNN demonstrated improvements across various offsets, inter-fiber distances and fiber diameters compared with their LR counterparts in terms of the traditional IQ metrics. Moreover, various degradation parameters showed different levels of impacts on the traditional IQ metrics of simulated LR and SR images. When increasing the offset of fibers, traditional IQ metrics of both LR and SR images decreased. Similar phenomena were observed when the inter-fiber distance and fiber diameters were increased. This is due to the more severe degradation model incorporated and thus DL-SR performs worse on images largely missing information when the network capacity or the number of training images were limited.

Fig. 9.

Fig. 9

Traditional IQ metrics show degradation of LR and limitation of improvement of SR images with reference to HR images: PSNR and SSIM values of LR (red) and SR (blue). The gray dashed line denotes a SSIM value of 0.95.

3.3. Endoscopists’ HRME Image Interpretation of DL-SR Images

Among the 120 paired HRME images and SR images included in the study, 42 images (35%) were non-neoplastic and 78 images (65%) were neoplastic. The diagnostic performance of endoscopists on HR and SR images is demonstrated in Figs. 10 and 11 for the summation of all four endoscopists’ reads. Overall, the task-based IQ of SR images was comparable to that of HR images when endoscopists interpreted them in a post-hoc setting. On the original HR images, the endoscopists achieved an overall accuracy of 70.2% (standard deviation [SD], 5.4%), a sensitivity of 73.7% (SD, 14.1%), and a specificity of 63.7% (SD, 12.8%). On the SR images, the endoscopists did not significantly change their accuracy (p=0.37), sensitivity (p=0.70), or specificity (p=0.07).

Fig. 10.

Fig. 10

Diagnostic performance of endoscopists’ reading on HR and SR microendoscopy images shown in box and whisker plot. The cross and horizontal line in the box represents mean and median values of the endoscopists’ response. The bottom and the top end from the whiskers are the minimum and maximum values, respectively.

Fig. 11.

Fig. 11

(a) Sensitivity and specificity plot of individual endoscopist diagnosis on HR and SR microendoscopic images with high and low confidence, respectively. (b) Confusion matrix of diagnostic performance of readers on HR and SR images with low and high confidence, respectively. TP, true positive; FP, false positive; FN, false negative; and TN, true negative.

Overall, endoscopists had high confidence in their image interpretation in a mean 51.1% (SD, 7.7%) of HR images and 48.5% of SR images (SD, 9.0%, p=0.68). Among high confidence images, endoscopists had no significant difference in their accuracy (76.0% versus 74.8%, p=0.73), sensitivity (84.6% versus 88.3%, p=0.45), or specificity (45.9% versus 30.2%, p=0.19) on SR images compared to HR counterparts. Similarly, for low-confidence images, endoscopists did not have a significant difference in their accuracy (64.2% versus 60.1%, p=0.39), sensitivity (58.1% to 63.7%, p=0.68), or specificity (73.3% versus 55.8%, p=0.14) on SR images compared to HR images.

4. Discussion

In this study, an endoscopic end-expandable optical fiber probe was proposed and evaluated for its ability to improve the field of imaging in EC screening. The novel concept of an end-expandable optical fiber probe was simulated by use of degradation model incorporating various probe parameters and was tested using HRME images of esophageal squamous tissue. We performed a virtual imaging trial to simulate LR microendoscopic image dataset from clinical HRME images and demonstrated the effectiveness of a DL algorithm to improve the perceptual IQ measures of LR images. Furthermore, the generated SR images were comparable to the original HRME images when interpreted by endoscopists in a diagnostics task.

The capability of the employed DL-SR algorithm to super-resolve an LR image of end-expandable optical fiber probe was proven for the first time here and should be highlighted because of its potential for further improvement and application. As expected, the quality of the SR images deteriorated when severe degradation models were considered. This indicated the significance of choosing tolerable parameters for future physical prototypes of the end-expandable optical fiber probe. We found that the diameter of a single fiber strand should be no more than 6  μm. The inter-fiber distance can be increased to up to 14 to 16  μm, which should be taken into account in the design and development of an optical fiber probe sleeve and a mechanism for controlling the inter-fiber distance. The fiber flexibility can permit offset of up to 4 to 6  μm. As a result, it will allow for increasing the diameter of an endoscopic sensor up to 4 to 5 mm instantly (the FOV area is 10 to 20 folds bigger than HRME) that can provide a dramatic improvement in esophageal screening efficiency. In the Supplemental Material and Video 1 (MP4, 6.03 MB [URL: https://doi.org/10.1117/1.JBO.29.4.046001.s1]), we demonstrate that the flexibility of single microfibers is very high and will not be limited by the expansion of the optical fiber brush in the applied range of microfiber curvation. The outer diameter of an expanded optical fiber brush and the corresponding simultaneously examined area depend on both the single fiber diameter and the range of inter-fiber distance.

The proposed concept of DL-SR image reconstruction for the case of limited sampling or low resolution can be applied to other optical methods of image acquisition. For example, the zoom mini lens were developed to enlarge microendoscopic FOV up to 1.15 mm.27,28 Given the principle of sparse image acquisition, the scanning sampling of zoom mini lens, its FOV and image resolution can be potentially enhanced.

Endoscopists had comparable diagnostic accuracy when interpreting original HRME images and DL-SR generated SR ones. While specificity could be improved, sensitivity was higher for SR images than for the original HR images. Our metrics of success focused on achieving comparable performance of human reads on SR images compared to that on HR images, which was achieved. Moreover, endoscopists had comparable level of confidence when interpreting SR images compared to the original HR images, which is a quantitative task-based assessment of the SR IQ. DL-SR methods that effectively estimate SR images, resembling the original HR images, is a robust strategy to increase the field of visualization during EC screening while maintaining accurate esophageal neoplasia detection by clinicians. More advanced deep-learning-based approaches for endoscopy super-resolution, such as those employing deformable transformer, contrastive adversarial learning, and zero-shot learning11,12,2932 should also be investigated for the end-expandable probe application in future studies.

This study evaluated the feasibility, imaging capability, and clinical performance of a hypothetical end-expandable optical fiber probe based on simulation studies using existing HRME images. Future in vivo studies will need to develop and test the fiber probe and its effectiveness in EC imaging. Using a larger dataset of the clinical microendoscopic images will potentially further advance the DL-SR performance. Another limitation of this study includes the rectangular orientation of the degradation model. A radial model of the optical fiber probe expansion will be applied to future studies.

5. Conclusion

We proposed a concept of an end-expandable optical fiber probe that would increase the field of visualization by up to 20-fold compared to a traditional fused HRME probes. We further validated the DL-SR generated images produced from the end-expandable optical fiber probe and found endoscopists’ interpretation of the SR images to be comparable to that of conventional HRME images. The proposed novel end-expandable optical fiber probe has the potential to enable high-yield endoscopic microscopy screening and even facilitate a screen-and-treat protocol for early EC treatment. Moreover, the proposed sparse image methodology will provide valuable guidance for future prototyping and the advancement of optical biopsy techniques over larger surface areas.

Supplementary Material

JBO_029_046001_SD001.pdf (905.5KB, pdf)
Download video file (5.9MB, mp4)

Acknowledgments

The authors acknowledge the contributions of Drs. Rebecca Richards-Kortum and Richard Schwarz from Rice University (Houston, Texas, United States) who provided the HRME instruments and helped to collect the clinical trial HRME images utilized here and who provided advice on the critical features of HRME images for data evaluation and image processing. Additionally, to the endoscopists of Baylor College of Medicine, a part of the HRME images were collected by Drs. Guiqi Wang, Fan Zhang, Hong Xu (Beijing and Changchun, China) whom we are grateful as well. This work was supported in part by the National Institutes for Health (NIH), National Cancer Institute (Grant No. R01 CA181275). Dr. Tan is supported by the National Institute of Diabetes and Digestive and Kidney Diseases of the NIH (Grant No. K23DK129776). Dr. Mark A. Anastasio was supported in part by NIH award no. EB031772 (subproject 6366). Preliminary results of this work were presented at SPIE Photonics West 2021 and published as an SPIE Proceedings paper.

Biography

Biographies of the authors are not available.

Contributor Information

Xiaohui Zhang, Email: xiaohui8@illinois.edu.

Mimi Tan, Email: mc2@bcm.edu.

Mansour Nabil, Email: Nabil.Mansour@bcm.edu.

Richa Shukla, Email: Richa.Shukla@bcm.edu.

Shaleen Vasavada, Email: Shaleen.Vasavada@bcm.edu.

Sharmila Anandasabapathy, Email: anandasa@bcm.edu.

Mark A. Anastasio, Email: maa@illinois.edu.

Elena Petrova, Email: elena.petrova.biocryst@gmail.com.

Disclosure

The authors declare no potential conflicts of interest.

Code and Data Availability

To be compliant with the requirements of the involved organizations, the raw datasets and code used in the current study can be obtained from the corresponding author upon request.

References

  • 1.Mansour N. M., Groth S. S., Anandasabapathy S., “Esophageal adenocarcinoma: screening, surveillance, and management,” Annu. Rev. Med. 68(1), 213–227 (2017). 10.1146/annurev-med-050715-104218 [DOI] [PubMed] [Google Scholar]
  • 2.Yang J., et al. , “Understanding esophageal cancer: the challenges and opportunities for the next decade,” Front. Oncol. 10, 1727 (2020). 10.3389/fonc.2020.01727 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bhushan S., Richards-Kortum R., Anandasabapathy S., “Progress and challenges of global high-resolution endoscopy,” Int. Arch. Intern. Med 4, 024 (2020). 10.23937/2643-4466/1710024 [DOI] [Google Scholar]
  • 4.Sharma P., et al. , “Real-time increased detection of neoplastic tissue in Barrett’s esophagus with probe-based confocal laser endomicroscopy: final results of an international multicenter, prospective, randomized, controlled trial,” Gastrointest. Endosc. 74(3), 465–472 (2011). 10.1016/j.gie.2011.04.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Tang Y., Anandasabapathy S., Richards-Kortum R., “Advances in optical gastrointestinal endoscopy: a technical review,” Mol. Oncol. 15(10), 2580–2599 (2021). 10.1002/1878-0261.12792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Bhushan S., et al. , “Photoacoustic imaging in gastroenterology: advances and needs,” in Photoacoustic Imaging - Principles, Advances and Applications, Gharieb R. R., Ed., IntechOpen; (2019). [Google Scholar]
  • 7.Polglase A. L., et al. , “A fluorescence confocal endomicroscope for in vivo microscopy of the upper- and the lower-GI tract,” Gastrointest. Endosc. 62(5), 686–695 (2005). 10.1016/j.gie.2005.05.021 [DOI] [PubMed] [Google Scholar]
  • 8.Anandasabapathy S., et al. , “An optical, endoscopic brush for high-yield diagnostics in esophageal cancer,” Proc. SPIE 11620, 116200B (2021). 10.1117/12.2583301 [DOI] [Google Scholar]
  • 9.Petrova E., Anandasabapathy S., Richards-Kortum R., “End expandable optical fiber bundle,” WO2022051741A2 (2022).
  • 10.Ravì D., et al. , “Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction,” Int. J. CARS 13(6), 917–924 (2018). 10.1007/s11548-018-1764-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Szczotka A. B., et al. , “Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy,” IEEE Trans. Med. Imaging 40(7), 1863–1874 (2021). 10.1109/TMI.2021.3067512 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Eadie M., et al. , “Fiber bundle image reconstruction using convolutional neural networks and bundle rotation in endomicroscopy,” Sensors 23(5), 2469 (2023). 10.3390/s23052469 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Dong C., et al. , “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). 10.1109/TPAMI.2015.2439281 [DOI] [PubMed] [Google Scholar]
  • 14.Ledig C., et al. , “Photo-realistic single image super-resolution using a generative adversarial network,” presented at Proc. IEEE Conf. Comput. Vision and Pattern Recognit., pp. 4681–4690 (2017). 10.1109/CVPR.2017.19 [DOI] [Google Scholar]
  • 15.Lai W.-S., et al. , “Fast and accurate image super-resolution with deep Laplacian pyramid networks,” IEEE Trans. Pattern Anal. Mach. Intell. 41(11), 2599–2613 (2018). [DOI] [PubMed] [Google Scholar]
  • 16.Jeong G., et al. , “A deep-learning-based image reconstruction method for USCT that employs multimodality inputs,” Proc. SPIE 12470, 124700M (2023). 10.1117/12.2654564 [DOI] [Google Scholar]
  • 17.Park S., et al. , “Stochastic three-dimensional numerical phantoms to enable computational studies in quantitative optoacoustic computed tomography of breast cancer,” J. Biomed. Opt. 28(6), 066002 (2023). 10.1117/1.JBO.28.6.066002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Protano M.-A., et al. , “Low-cost high-resolution microendoscopy for the detection of esophageal squamous cell neoplasia: an international trial,” Gastroenterology 149(2), 321–329 (2015). 10.1053/j.gastro.2015.04.055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Tan M. C., et al. , “Automated software-assisted diagnosis of esophageal squamous cell neoplasia using high-resolution microendoscopy,” Gastrointest. Endosc. 93(4), 831–838.e2 (2021). 10.1016/j.gie.2020.07.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Wang Z., Chen J., Hoi S. C., “Deep learning for image super-resolution: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 43, 3365–3387 (2020). 10.1109/TPAMI.2020.2982166 [DOI] [PubMed] [Google Scholar]
  • 21.Yang W., et al. , “Deep learning for single image super-resolution: a brief review,” IEEE Trans. Multimedia 21(12), 3106–3121 (2019). 10.1109/TMM.2019.2919431 [DOI] [Google Scholar]
  • 22.Zhang X., et al. , “Impact of deep learning-based image super-resolution on binary signal detection,” J. Med. Imaging 8(6), 065501 (2021). 10.1117/1.JMI.8.6.065501 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Sengupta S., et al. , “DeSupGAN: multi-scale feature averaging generative adversarial network for simultaneous de-blurring and super-resolution of retinal fundus images,” Lect. Notes Comput. Sci. 12069, 32–41 (2020). 10.1007/978-3-030-63419-3_4 [DOI] [Google Scholar]
  • 24.Kelkar V. A., et al. , “Task-based evaluation of deep image super-resolution in medical imaging,” Proc. SPIE 11599, 115990X (2021). 10.1117/12.2582011 [DOI] [Google Scholar]
  • 25.Saharia C., et al. , “Image super-resolution via iterative refinement,” IEEE Trans. Pattern Anal. Mach. Intell. 45(4), 4713–4726 (2022). [DOI] [PubMed] [Google Scholar]
  • 26.Kingma D. P., Ba J., “Adam: a method for stochastic optimization,” arXiv:1412.6980 (2014).
  • 27.Ouzounov D. G., et al. , “Dual modality endomicroscope with optical zoom capability,” Biomed. Opt. Express 4(9), 1494–1503 (2013). 10.1364/BOE.4.001494 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Ouzounov D. G., et al. , “Miniature varifocal objective lens for endomicroscopy,” Opt. Lett. 38(16), 3103–3106 (2013). 10.1364/OL.38.003103 [DOI] [PubMed] [Google Scholar]
  • 29.Song X., et al. , “Deformable transformer for endoscopic video super-resolution,” Biomed. Signal Process. Control 77, 103827 (2022). 10.1016/j.bspc.2022.103827 [DOI] [Google Scholar]
  • 30.Zhang C., Gu Y., Yang G.-Z., “Contrastive adversarial learning for endomicroscopy imaging super-resolution,” IEEE J. Biomed. Health Inf. 27(8), 3994–4005 (2023). 10.1109/JBHI.2023.3275563 [DOI] [PubMed] [Google Scholar]
  • 31.Chen W., et al. , “Dynamic depth-aware network for endoscopy super-resolution,” IEEE J. Biomed. Health Inf. 26(10), 5189–5200 (2022). 10.1109/JBHI.2022.3188878 [DOI] [PubMed] [Google Scholar]
  • 32.Ravì D., et al. , “Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy,” Med. Image Anal. 53, 123–131 (2019). 10.1016/j.media.2019.01.011 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

JBO_029_046001_SD001.pdf (905.5KB, pdf)
Download video file (5.9MB, mp4)

Data Availability Statement

To be compliant with the requirements of the involved organizations, the raw datasets and code used in the current study can be obtained from the corresponding author upon request.


Articles from Journal of Biomedical Optics are provided here courtesy of Society of Photo-Optical Instrumentation Engineers

RESOURCES