Application and progress of artificial intelligence technology in the segmentation of hyperreflective foci in OCT images for ophthalmic disease research

Abstract

With the advancement of retinal imaging, hyperreflective foci (HRF) on optical coherence tomography (OCT) images have gained significant attention as potential biological biomarkers for retinal neuroinflammation. However, these biomarkers, represented by HRF, present pose challenges in terms of localization, quantification, and require substantial time and resources. In recent years, the progress and utilization of artificial intelligence (AI) have provided powerful tools for the analysis of biological markers. AI technology enables use machine learning (ML), deep learning (DL) and other technologies to precise characterization of changes in biological biomarkers during disease progression and facilitates quantitative assessments. Based on ophthalmic images, AI has significant implications for early screening, diagnostic grading, treatment efficacy evaluation, treatment recommendations, and prognosis development in common ophthalmic diseases. Moreover, it will help reduce the reliance of the healthcare system on human labor, which has the potential to simplify and expedite clinical trials, enhance the reliability and professionalism of disease management, and improve the prediction of adverse events. This article offers a comprehensive review of the application of AI in combination with HRF on OCT images in ophthalmic diseases including age-related macular degeneration (AMD), diabetic macular edema (DME), retinal vein occlusion (RVO) and other retinal diseases and presents prospects for their utilization.

OVERVIEW OF ARTIFICIAL INTELLIGENCE TECHNOLOGY

In 1955, John McCarthy coined the term “artificial intelligence” (AI) to refer to the science and engineering involved in creating intelligent machines^[1]. Initially, AI was primarily used for solving complex mathematical problems. However, in recent years, with the rapid advancement of information technology in the medical field, there has been a growing focus on utilizing AI to assist clinical researchers in studying diseases.

AI is a discipline that encompasses the development of theories, methods, techniques, and applications aimed at simulating, extending, and enhancing human intelligence. Its objective is to emulate human thinking and utilize efficient algorithms to uncover patterns from vast amounts of data, enabling the summarization, interpretation, prediction, and analysis of phenomena and unknown information^[2]–^[3]. With the progress of medical equipment and medical imaging technology, the digitization and storage of medical data have become possible. AI has emerged as one of the most influential information technologies, garnering extensive attention and application in the field of medicine^[4]. Machine learning (ML), an integral part of AI, comprises algorithms specifically designed for biological data and biological problems^[5]. Traditional ML algorithms include support vector machines and random forests. Deep learning (DL), a subfield of ML, has the capability to integrate large datasets and learn complex relationships^[6], thereby assisting clinical practitioners in their work. DL includes research models such as convolutional neural networks^[7] and artificial neural networks^[8]. In the research and application of images, people tend to be only interested in certain parts of the image, the significant step in computer vision technology is image segmentation, which is an important part of image semantic understanding^[9]. Medical image segmentation is a challenging step in the construction of AI models, and it exemplifies the perfect integration of the medical and computer fields. Medical images often face issues such as poor contrasts, noise interference, unclear boundaries, and poor visual effects^[10]. Effective image segmentation techniques can extract clear target regions efficiently. They are essential for lesion quantification and choice of subsequent treatment methods. Traditional image segmentation methods include threshold method, edge detection method, region growing segmentation, and Cluster analysis. Another class of segmentation methods is based on Convolutional Neural Networks, because of its excellent feature extraction capabilities, it greatly improves the performance of medical image segmentation. The magnitude of the training set significantly impacts the design and functionality of the model recognition system. An inadequate training set size could potentially lead to significant model bias. Both the size of the training and test sets contribute to the variance of model performance assessment^[11]. Empirical research indicates that an adequate sample size is essential for the development and validation of AI predictive models. However, in the context of medical imaging, a definitive approach to determining the sample size of AI training sets has yet to be established. It is generally conceded that a sufficiently large sample size is critical for robust development and accurate validation of AI-based predictive models^[12].

In recent years, the application of AI in the field of ophthalmology has undergone rapid development. It has made significant contributions to the study of diseases such as diabetic retinopathy (DR)^[13], age-related macular degeneration (AMD)^[14], glaucoma^[15]–^[16], and retinopathy of prematurity (ROP)^[17].

In addition, AI also includes the detection of genes related to ophthalmic diseases, ocular metabolites, and pathology ocular metabolites^[18]. The research on AI demonstrates a diversification of diseases and an in-depth exploration, which holds great significance for disease screening, diagnosis, treatment, and prediction (Figure 1).

Figure 1. OCT images submitted to the AI model for feature extraction and analysis.

OCT: Optical coherence tomography; AI: Artificial intelligence.

OVERVIEW OF HYPERREFLECTIVE FOCI

With the advancement of ophthalmic imaging, continuous innovation has occurred in the techniques of spectral-domain optical coherence tomography (SD-OCT) and optical coherence tomography angiography (OCTA), facilitating the acquisition of high-resolution retinal anatomical microstructure images and imaging features of different layers of capillaries in clinical practice. Hyperreflective foci (HRF) are biological markers of retinal neuroinflammation that reveal the prognosis and clinical significance of various ophthalmic diseases. HRF are defined as discrete, circular, scattered, and well-demarcated high-signal lesions observed on OCT^[19]. Research has identified the presence of HRF in diseases such as diabetic macular edema (DME)^[20]–^[21], AMD^[22], retinal vascular occlusions^[23]–^[24], central serous chorioretinopathy (CSC)^[25], and degenerative retinal diseases^[26]. The exact clinical and pathological mechanisms of HRF are not yet fully understood, but they are closely related to lipid exudation and translocated retinal pigment epithelial cells in DME, as well as macrophages, microglial cells, and degenerated photoreceptor cells in AMD^[27]. As our understanding of HRF continues to deepen, their significance in predicting disease progression, treatment outcomes, and visual prognosis becomes increasingly significant.

ADVANCEMENTS IN AI RESEARCH ON HYPERREFLECTIVE FOCI IN OPHTHALMIC DISEASES

Age-Related Macular Degeneration

AMD is a major cause of severe vision impairment among the elderly population worldwide. Given the ongoing increase in the aging population, it is projected that the number of individuals affected by this condition will reach 288 million by the year 2040^[28]. OCT has emerged as the primary imaging modality for the initial evaluation, subsequent treatment, and follow-up of AMD. The integration of AI into OCT images processing has significantly enhanced the analysis of OCT images and the identification of specific biological markers. HRF are of particular significance observed in OCT images of AMD patients, which are believed to primarily comprise lipid accumulation, microglia cells, and migrated or transdifferentiated retinal pigment epithelial cells^[29]. HRF play a critical role in predicting the progression of AMD.

Bogunovic et al^[30] employed a convolutional neural network to automatically segment HRF situated between different retinal layers in order to predict intermediate AMD progression. Throughout the study, AMD patients underwent follow-up every three months, with their progress recorded through OCT imaging. The average duration of follow-up was 37.86±13.8mo. The AI model's predictive performance was evaluated using the area under the curve (AUC) for different follow-up periods. Within the initial two years, the predicted AUC reached 0.75. Additionally, the researchers identified a correlation between the volume of HRF in the outer nuclear layer and the regression of drusen. In 2018, researchers utilized ML algorithms in combination with biological markers, including HRF, drusen, and retinal morphology observed on OCT, to develop a model capable of estimating early AMD progression. The study encompassed a total of 495 AMD-affected eyes, of which 159 eyes converted to advanced AMD within two years, 114 eyes advanced to choroidal neovascularization, and another 45 eyes progressed to geographic atrophy. The study highlighted that key quantitative features associated with disease progression were outer retinal layer thickness, HRF, and drusen^[31].

Waldstein et al^[32] conducted a two-year follow-up study involving 512 patients diagnosed with AMD, during which a total of 8529 OCT images were collected. DL algorithms were utilized for the precise localization and quantitative analysis of drusen and HRF on OCT images. The findings revealed that eyes that progressed to advanced AMD exhibited larger volume of drusen and HRF within the eye. Specifically, in patients who progressed to macular atrophy, the HRF were observed to cluster around 0.5 µm from the center of the macula^[32]. Moreover, it was confirmed that HRF serve as a significant negative prognostic indicator strongly associated with poor outcomes in best-corrected visual acuity^[33]. The evident role of AI technology in the diagnosis of AMD patients not only enhances disease detection but also provides novel avenues for treatment and follow-up.

Diabetic Macular Edema

DR is a prevalent microvascular complication of diabetes mellitus, often causing severe visual impairment and being a leading cause of blindness among the working-age population^[34]. DME, a manifestation of DR, can occur at any stage of the disease and lead to a notable decline in visual acuity. Despite the high prevalence and the consequential impact on vision, early screening plays a vital role in timely diagnosis and treatment and offers substantial benefits to most patients. OCT is a non-contact and rapid imaging technique that provides high-resolution retinal microstructural images. Its utilization significantly enhances the detection rate and accuracy of DME. Furthermore, OCT generates ample data to support research in the field of AI.

Yu et al^[35] developed a deep convolutional neural network model using the GoogLeNet algorithm to automatically identify HRF in the retina. Their model demonstrated improved accuracy in segmenting interlayer HRF compared to traditional models, providing valuable assistance in clinical diagnosis. Schmidt et al^[36] utilized a dataset of 2596 OCT B-scan images from seven patients to create a model based on the Blob algorithm. This model accurately identified HRF between the outer plexiform layer and the retinal pigment epithelium, achieving an accuracy, sensitivity, and specificity of 96.3%, 88.4%, and 97.5%, respectively. Wei et al^[37] proposed the DBR neural network for precise segmentation of OCT images, exhibiting a processing speed of 57ms per OCT image, which is 150ms faster than alternative methods, greatly enhancing clinical efficiency. Since the segmentation of HRF in retinal SD-OCT volumes with low contrast features posed challenges, researchers had made advancements by improving the existing 3D U-Net algorithm and employing image enhancement techniques to generate additional enhanced images. Experimental results had demonstrated the effectiveness of this approach in achieving rapid segmentation and reliable outcomes, thereby facilitating clinical diagnosis and disease monitoring^[38].

In the diagnosis and prognosis of DME, various biological markers are considered crucial, including intraretinal fluid and subretinal fluid volume, external limiting membrane and ellipsoid zone integrity, and the presence of HRF^[39]. Midena et al^[40] employed an AI model to automatically segment these biological markers, including HRF, intraretinal fluid, and subretinal fluid, in OCT images for both quantitative and visual analysis. The results were compared with manual annotations by ophthalmology experts, revealing nearly identical outcomes between the AI software and clinical evaluation. The automated quantification of IRF, subretinal fluid (SRF), extermal limiting membranes (ELM), and ellipsoid zone (EZ) displayed an accuracy ranging from 94.7% to 95.7%. Notably, the intraclass correlation coefficient for counting HRF was 0.97, indicating excellent reliability. In another study, DL algorithms were employed to develop a model capable of classifying macular edema resulting from DR, retinal vein occlusion (RVO), and AMD by identifying biological markers such as IRF, SRF, and HRF. This model exhibited high accuracy and reliability, comparable to expert interpretation ^[41].

Retinal Vein Occlusion

RVO stands as one of the most prevalent retinal vascular disorders, often affecting elderly male patients^[42]. The pathogenesis of RVO is closely linked to factors such as vascular endothelial injury, hemodynamic changes, intraocular pressure, and localized eye compression^[43]. AI has emerged as a valuable clinical tool for the early screening of RVO. Nagasato et al^[44] proposed the use of ultra-widefield fundus photographs in combination with DL algorithms for the accurate early diagnosis of branch RVO, offering particular benefits in regions with limited access to ophthalmic medical centers. Additionally, OCT had proven to be highly effective in diagnosing and managing RVO. However, manually identifying biological markers during OCT scanning could be error-prone and time-consuming. To address this challenge, researchers had proposed the application of the ResU-Net algorithm for segmenting HRF in OCT images. ResU-Nets demonstrated superior accuracy in detecting HRF and could be applied to identify HRF in various retinal diseases, beyond the specific diseases for which the models were originally trained^[45].

Other Retinal Diseases

Uveitis and CSC represent common ophthalmic diseases. CSC, predominantly affecting male patients, is characterized by neurosensory retinal detachment in the central macular area caused by defective retinal pigment epithelial leakage^[46]. Current treatment options for CSC encompass observation, laser therapy, photodynamic therapy^[47], and oral eplerenone^[48]. However, a definitive consensus on the optimal treatment strategy is yet to be reached. Consequently, researchers had employed decision tree algorithms to construct an AI model for assessing the treatment outcomes of CSC. The study indicated that HRF and age play decisive roles in the prognosis of CSC. Patients with retinal HRF benefit from oral eplerenone, whereas those with choroidal HRF have improved prognoses with PDT treatment. Furthermore, researchers had explored the correlation between the risk of macular neovascularization formation and biological markers such as retinal pigment epithelium thickness and retinal thickness^[49].

To address the quantification of vitreous HRF (vHF) in uveitis patients, Lee et al^[50] developed a DL algorithm based on OCT images. The training dataset comprised 648 B-scan OCT images. The validation results of the DL model demonstrated good agreement with the manually obtained results by clinical doctors, with a Dice coefficient of 0.69. This signified the reliable quantification ability of the AI model and the time and effort savings facilitated by the efficient image segmentation technique, reducing the need for manual analysis.

CONCLUSION AND OUTLOOK

AI plays a pivotal role in the detection, diagnosis, grading, and classification of various ophthalmic conditions, particularly in retinal diseases. This emerging technology offers significant advantages in automatic screening and diagnosis, while also mitigating the biases associated with manual measurements and counting. By employing AI visualization and quantification techniques, common biological markers in imaging studies can be accurately analyzed. Furthermore, AI technology presents valuable opportunities for patients in remote areas with limited access to medical resources, thus reducing the likelihood of delayed treatment and alleviating the burden on the healthcare system. Numerous studies have demonstrated the high accuracy and sensitivity of AI.

However, the utilization of AI technology has certain limitations. These limitations can be divided into two aspects: AI model development and clinical application. In terms of AI model development, limitations often exist in the following areas: 1) Image quality: patients with cataracts and other diseases leading to unclear media or poor patient cooperation can degrade image quality. Additionally, the quality of images can also be influenced by the imaging equipment and operators. 2) Sample size: the accuracy of AI models is closely related to the size of the collected sample, with larger sample sizes leading to higher model accuracy. 3) Manual annotation: before training AI models, images in the dataset need to be manually annotated for the region of interest. However, due to the small size of HRF, which can induce bias, and have a significant impact on the performance of AI models. On the other hand, limitations in clinical application include: 1) Patient heterogeneity: the samples used for developing AI models often focus on a specific ethnicity or a particular region. Therefore, differences between patients, such as age, gender, ethnicity, and region, occur biases in results and affects the performance of AI models^[51]. 2) Single target diseases: current research suggests that a specific AI model is often designed to detect a particular disease, which means the possibility of missed diagnoses in clinical applications^[52]–^[53]. 3) Disease restrictions: the construction of AI models often requires a large amount of data support. Consequently, constructing AI models for rare diseases can be challenging and result in lower accuracy and sensitivity^[4]. According to the above, researchers should strive to increase the sample size as much as possible to reduce biases. Additionally, manual annotation of images in the dataset often necessitates the involvement of experts in the relevant diseases to ensure data validity through repeated checks. Data collection and management of ophthalmic examinations, model development, clinical trials, and clinical applications are the four keys of ophthalmic AI. Do each step well, ensuring that the models developed in the study have high quality, reliability, and stability^[54].

In conclusion, AI has garnered significant research interest in the realm of ophthalmic image processing and shows promising development prospects. Its accuracy has reached a level on par with clinical experts. Nonetheless, AI does have certain limitations. It is anticipated that researchers will continue to delve into and enhance AI systems in future studies, aiming to facilitate their extensive application in the medical field.