Abstract.
Guest editors Behrouz Shabestri, Mark Anastasio, Baowei Fei, and Frédéric Leblond provide an overview of the JBO Special Series on Artificial Intelligence Machine Learning in Biomedical Optics.
Artificial Intelligence (AI) methods, including machine learning (ML) and deep learning (DL), are quickly evolving, and impacting a very wide range of scientific endeavors. Biomedical optics is no exception and AI methods are currently transforming our discipline on an almost daily basis. From changing data acquisition1,2 and image reconstruction methods,3 to segmentation and interpretation of optical images,4 AI methods are providing improved solutions to established problems and enabling new problems to be addressed.
Structured light can be combined with AI methods to probe and interpret the interaction of light with biological tissues. For example, the coupling of AI methods with hyperspectral and multispectral systems can enable the detection of specific molecular signatures in tissue, cells, and biofluids.5,6 Supervised ML/DL methods are well-suited for this purpose, since they can implicitly learn high-dimensional image statistics and complicated mappings that describe optimal decision strategies for a variety of inferences of relevance to basic science and clinical applications.
Enhancing advanced optical methods with AI will enable the clinical translation of new optical sensing and imaging technologies. Label free optical imaging, such as stimulated Raman histology, hyperspectral imaging, and convolutional neural networks (CNNs), has been successfully employed for intraoperative automated brain tumor diagnosis with near real-time detection.7,8 Integrating ML/DL methods with optical methods such as coherent anti-Stokes Raman scattering imaging, optical colonoscopy and fluorescence lifetime imaging has shown to be effective in the differential diagnosis of lung cancer,9 colorectal cancer,10 and cervical neoplasia,11 respectively. Another AI-enabled game-changer will be the use of DL methods for computational staining of label-free optical images, resulting in all-digital histopathology.12–14
In clinical decision making, where accuracy and timing can be critical, spatial frequency domain imaging coupled with ML has been employed for predicting the severity of burn injuries.15 The combination of multi-photon imaging with ML/DL has further enabled improved lymphedema diagnosis,16 skin cancer screening17 and atopic dermatitis.18 ML combined with emerging feature engineering approaches has become the mainstay in tissue, cells, and biofluids interrogation in spectroscopic methods. Examples of such applications range from neurosurgical guidance using spontaneous Raman spectroscopy for cancer detection19 to detection of aggressive variants of prostate cancer in pathology using Raman micro-spectroscopy.20
Merging optical coherence tomography (OCT) imaging with AI provides a unique opportunity to analyze this plethora of information and assist in making clinical decisions in the field of ophthalmology with applications in retinal imaging,21 glaucoma22 and age-related macular degeneration.23
Most recently, AI methods are proving to be invaluable for a variety of tasks related to the detection and management of COVID-19.24–26 Combining AI with optical breathalyzers may yield a rapid and accurate test for COVID-19, which is currently lacking and greatly needed.
This JBO special series brings together late breaking research that describe the use of artificial intelligence in biophotonic applications, with an emphasis on ML and DL approaches. The series highlights the important role that ML and DL methods are playing in accelerating the development of innovative biophotonic technologies. This series is timely, for it comes as a growing number of the biomedical optics scientific community are starting to tackle the multiple challenges associated with the responsible adoption of AI methods. Issues such as robustness, reliability, and interpretability remain largely unaddressed but are critical for safe and effective deployment of AI-enabled biophotonic imaging and sensing systems. We hope you enjoy this special series, which includes the following twelve articles:
C. Canavesi, A. Cogliati, and H. B. Hindman, “Unbiased corneal tissue analysis using Gabor-domain optical coherence microscopy and machine learning for automatic segmentation of corneal endothelial cells,” doi 10.1117/1.JBO.25.9.092902
A. Hauptmann and B. T. Cox, “Deep learning in photoacoustic tomography: current approaches and future directions,” doi 10.1117/1.JBO.25.11.112903
B. O. L. Mellors et al., “Applications of compressive sensing in spatial frequency domain imaging,” doi 10.1117/1.JBO.25.11.112904
I. Fredriksson, M. Larsson, and T. Strömberg, “Machine learning for direct oxygen saturation and hemoglobin concentration assessment using diffuse reflectance spectroscopy,” doi 10.1117/1.JBO.25.11.112905
D. S. Gareau et al., “Deep learning-level melanoma detection by interpretable machine learning and imaging biomarker cues,” doi 10.1117/1.JBO.25.11.112906
M. Chen and N. Durr, “Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning,” doi 10.1117/1.JBO.25.11.112907
B. Lyu et al., “Domain adaptation for robust workload level alignment between sessions and subjects using fNIRS,” doi 10.1117/1.JBO.26.2.022908
S. Guo et al., “FLIM data analysis based on Laguerre polynomial decomposition and machine-learning,” doi 10.1117/1.JBO.26.2.022909
M. S. Durkee et al., “Quantifying the effects of biopsy fixation and staining panel design on automatic instance segmentation of immune cells in human lupus nephritis,” doi 10.1117/1.JBO.26.2.022910
F. Daoust et al., “Handheld macroscopic Raman spectroscopy imaging instrument for machine learning based molecular tissue margins characterization,” doi 10.1117/1.JBO.26.2.022911
M. H. Nguyen et al., “Machine learning to extract physiological parameters from multispectral diffuse reflectance spectroscopy,” doi 10.1117/1.JBO.26.5.052912
B. X. Guan et al., “Human embryonic stem cell classification: random network with autoencoded feature extractor,” doi 10.1117/1.JBO.26.5.052913.
Biographies
Behrouz Shabestari is the acting director of the Division of Health Informatics Technologies and director of the NIBIB National Technology Centers Program. He directs the NIBIB programs in optical imaging and spectroscopy, and for x-ray, electron, ion beam, and computed tomography (CT). He joined the NIBIB in 2015, after 12 years as a scientific review officer at the NIH Center for Scientific Review’s Surgical Sciences, Biomedical Imaging, and Bioengineering Integrated Review Group. There, he was responsible for the review of applications in the development of methods for a wide variety of medical imaging modalities and bioengineering, including SPECT, PET, MRI/MRS, ultrasound, CT, photonics, image-guided surgery, CAD, image recognition algorithms, as well as hybrid approaches. He has extensive experience in the area of industrial and medical imaging.
Mark A. Anastasio is the Donald Biggar Willett Professor in Engineering at University of Illinois Urbana–Champaign (UIUC), where he also heads the Department of Bioengineering. His lab within the Grainger College of Engineering at UIUC, the Computational Imaging Science Laboratory, performs research in computational and theoretical image science and pursues the advancement of emerging imaging methods. He is an elected fellow of the International Society for Optics and Photonics (SPIE) and of the American Institute for Medical and Biological Engineering (AIMBE).
Baowei Fei is a professor of bioengineering, the Cecil H. and Ida Green Chair in Systems Biology Science, and Dean’s Fellow at the Erik Jonsson School of Engineering and Computer Science at the University of Texas at Dallas. He is also a professor of radiology at UT Southwest Medical Center. He is the director of the Quantitative BioImaging Laboratory (https://fei-lab.org/). He is the director of the Center for Imaging and Surgical Innovation at UT Dallas and UT Southwestern Medical Center. He is a national leader in quantitative imaging and image-guided interventions. He served as conference chair for the International Conference of SPIE Medical Imaging—Image-Guided Procedures, Robotics Interventions, and Modeling from 2017 to 2020. He is a fellow of SPIE and the AIMBE.
Frédéric Leblond is a professor in the Department of Engineering Physics at Polytechnique Montréal, where he heads the Optical Radiology Laboratory. He works mainly in biomedical optics (including diffuse optics and spectroscopy), designing new surgical and pathology methods, enhancing medical imaging, and studying light propagation in biological tissues. He is the co-founder and was—until 2020—technical director of ODS Medical Inc., which is tasked with commercialization of his Raman-spectroscopy-based cancer-cell detection device. He is currently working with a number of industrial partners on development of several medical imaging techniques, fiber optical systems, and software. He also holds several patents. As his work also involves human subjects and is greatly useful to medical personnel, he has collaborative projects with many hospitals across North America.
References
- 1.Sitzmann V., et al. , “End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging,” ACM Trans. Graphics 37(4), 1–13 (2018). 10.1145/3197517.3201333 [DOI] [Google Scholar]
- 2.Muthumbi A., et al. , “Learned sensing: jointly optimized microscope hardware for accurate image classification,” Biomed. Opt. Express 10(12), 6351–6369 (2019). 10.1364/BOE.10.006351 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Belthangady C., Royer L. A., “Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction,” Nat. Methods 16(12), 1215–1225 (2019). 10.1038/s41592-019-0458-z [DOI] [PubMed] [Google Scholar]
- 4.Lee C. S., et al. , “Deep-learning based, automated segmentation of macular edema in optical coherence tomography,” Biomed. Opt. Express 8(7), 3440–3448 (2017). 10.1364/BOE.8.003440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ortega S., et al. , “Hyperspectral imaging for the detection of glioblastoma tumor cells in H&E slides using convolutional neural networks,” Sensors, 20(7), 1911 (2020). 10.3390/s20071911 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Halicek M., et al. , “Hyperspectral imaging of head and neck squamous cell carcinoma for cancer margin detection in surgical specimens from 102 patients using deep learning,” Cancers 11(9), 1367 (2019). 10.3390/cancers11091367 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hollon T. C., et al. , “Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks,” Nat. Med. 26(1), 52–58 (2020). 10.1038/s41591-019-0715-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Fabelo H., et al. , “Deep learning-based framework for in vivo identification of glioblastoma tumor using hyperspectral images of human brain,” Sensors 19(4), 920 (2019). 10.3390/s19040920 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sheng W., et al. , “Combining deep learning and coherent anti-Stokes Raman scattering imaging for automated differential diagnosis of lung cancer,” J. Biomed. Opt. 22, 106017 (2017). 10.1117/1.JBO.22.10.106017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zhou D., et al. , “Diagnostic evaluation of a deep learning model for optical diagnosis of colorectal cancer,” Nat. Commun. 11, 2961 (2020). 10.1038/s41467-020-16777-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gu J., et al. , “Quantitative diagnosis of cervical neoplasia using fluorescence lifetime imaging on haematoxylin and eosin stained tissue sections,” J Biophotonics 7(7), 483–491 (2014). 10.1002/jbio.201200202 [DOI] [PubMed] [Google Scholar]
- 12.Schnell M., et al. , “All-digital histopathology by infrared-optical hybrid microscopy,” Proc. Natl. Acad. Sci. USA 117(7), 3388–3396 (2020). 10.1073/pnas.1912400117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kandel M. E., et al. , “Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments,” Nat. Commun. 11, 6256 (2020). 10.1038/s41467-020-20062-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Halicek M., et al. , “Head and neck cancer detection in digitized whole-slide histology using convolutional neural networks,” Sci. Rep. 9, 14043 (2019). 10.1038/s41598-019-50313-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rowland R., et al. , “Burn wound classification model using spatial frequency-domain imaging and machine learning,” J. Biomed. Opt. 24, 056007 (2019). 10.1117/1.JBO.24.5.056007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kistenev Y. V., et al. , “Application of multiphoton imaging and machine learning to lymphedema tissue analysis,” Biomed. Opt. Express 10(7), 3353–3368 (2019). 10.1364/BOE.10.003353 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Huttunen M. J., et al. , “Multiphoton microscopy of the dermoepidermal junction and automated identification of dysplastic tissues with deep learning,” Biomed. Opt. Express 11(1), 186–199 (2020). 10.1364/BOE.11.000186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Guimarães P., et al. , “Artificial intelligence in multiphoton tomography: atopic dermatitis diagnosis,” Sci. Rep. 10(1), 7968 (2020). 10.1038/s41598-020-64937-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lemoine É., et al. , “Feature engineering applied to intraoperative in vivo Raman spectroscopy sheds light on molecular processes in brain cancer: a retrospective study of 65 patients,” Analyst 144(22), 6517–6532 (2019). 10.1039/C9AN01144G [DOI] [PubMed] [Google Scholar]
- 20.Grosset A.-A., et al. , “Identification of intraductal carcinoma of the prostate on tissue specimens using Raman micro-spectroscopy: a diagnostic accuracy case–control study with multicohort validation,” PLoS Med. 17(8). e1003281 (2020). 10.1371/journal.pmed.1003281 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Lee C. S., et al. , “Generating retinal flow maps from structural optical coherence tomography with artificial intelligence,” Sci. Rep. 9(1), 5694 (2019). 10.1038/s41598-019-42042-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Fu H., et al. , “A deep learning system for automated angle-closure detection in anterior segment optical coherence tomography images,” Am. J. Ophthalmol. 203, 37–45 (2019). 10.1016/j.ajo.2019.02.028 [DOI] [PubMed] [Google Scholar]
- 23.Schlegl T., et al. , “Fully automated detection and quantification of macular fluid in OCT using deep learning,” Ophthalmology 125(4), 549–558 (2018). 10.1016/j.ophtha.2017.10.031 [DOI] [PubMed] [Google Scholar]
- 24.Vaishya R., et al. , “Artificial intelligence (AI) applications for COVID-19 pandemic,” Diab. Metab. Syndrome Clin. Res. Rev. 14(4), 337–339 (2020). 10.1016/j.dsx.2020.04.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Shi F., et al. , “Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19,” IEEE Rev. Biomed. Eng. 14, 4–15 (2021). 10.1109/RBME.2020.2987975 [DOI] [PubMed] [Google Scholar]
- 26.Carlomagno C., et al. , “COVID-19 salivary Raman fingerprint: innovative approach for the detection of current and past SARS-CoV-2 infections,” Sci. Rep. 11, 4943 (2021). 10.1038/s41598-021-84565-3 [DOI] [PMC free article] [PubMed] [Google Scholar]