Abstract
Objective
OCT angiography (OCTA) images present challenges for clinical and research use due to variability and noise. The aim was to develop artificial intelligence models for OCTA image quality evaluation using the Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights (AI-READI) data set.
Design
Cross-sectional study.
Subjects
Six thousand two hundred sixty-nine OCTA 6 × 6-mm and 325 12 × 12-mm macula-centered photographs of the superficial vascular plexus from 1067 AI-READI study participants, and 1100 6 × 6-mm OCTA photographs from 539 patients from the Massachusetts Eye and Ear Infirmary (MEEI).
Methods
All photographs were labeled as acceptable or poor quality by two ophthalmologists and five medical students. Predefined training and validation sets were used to fine-tune four deep learning (DL) models (ResNet, EfficientNet, Vision Transformer [ViT], and ConvNeXt v2) via hyperparameter grid search.
Main Outcome Measures
Model performance on all photographs from the AI-READI and MEEI test sets were assessed by accuracy, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) scores.
Results
The accuracy/AUROC of the fine-tuned ResNet, EfficientNet, ViT, and ConvNeXt v2 models on the 6 × 6-mm AI-READI test set were 83.5%/0.904, 82.5%/0.904, 83.5%/0.908%, and 82.9%/0.913, with sensitivities/specificities of 77.9%/86.1%, 73.4%/89.9%, 69.2%/92.7%, and 79.8%/84.2%, respectively. The accuracy/AUROC of these models on the 6 × 6-mm MEEI test set were 91.5%/0.925, 94.3%/0.974, 93.9%/0.974%, and 88.0%/0.957, with sensitivities/specificities of 73.2%/98.4%, 84.5%/97.3%, 86.8%/97.2%, and 84.4%/92.7%, respectively. The accuracies/AUROC for the 12 × 12-mm photographs were 72.3%/0.775, 76.4%/0.775, 66.2%/0.979%, and 83.7%/0.834, with sensitivities/specificities of 95.7%/54.5%, 95.7%/57.8%, 97.9%/42.2%, and 96.4%/74.1%, respectively.
Conclusions
The AI-READI data set contains heterogeneous data usable for high-performing DL models for OCTA image quality assessment. These models were generalizable across institutions, cameras, and image sizes and may be used to rapidly screen OCTA image quality for use in future clinical studies.
Financial Disclosure(s)
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Keywords: Artificial intelligence, OCT angiography, Deep learning, Imaging, Image quality.
Advances in OCT angiography (OCTA) have facilitated noninvasive imaging of retinal vasculature via acquisition of sequential OCT scans and postacquisition processing.1 This technology has transformed retinal vascular evaluation and has facilitated identifying novel clinical biomarkers in diabetic retinopathy,2, 3, 4, 5 age-related macular degeneration,4,6,7 sickle cell retinopathy,8,9 retinal vein occlusions,10,11 among other choroidal and retinal diseases,12,13 and glaucoma.14,15 Due to its noninvasive nature, OCTA offers key advantages over traditional imaging methods such as fluorescein angiography and indocyanine green angiography.
Variations in image quality remain a key barrier to widespread OCTA adoption. There are several explanations for the high variability in OCTA image quality, including challenges with high-resolution microvasculature imaging, motion artifacts, and artifacts from pupil size and media opacity.15 Although several commercially available OCTA cameras provide proprietary image quality metrics (i.e., signal strength or signal strength index), these metrics do not necessarily correlate with or distinguish artifacts that may affect image quality.5,15,16 Recent studies have explored artificial intelligence (AI), and more specifically deep learning (DL), in automated image quality assessment of the posterior segment. Prior work has focused on single-center development of models for assessment of 3 × 3-mm17 and 6 × 6-mm en face OCTA scans of the macula16,18 and optic nerve.19,20 Additional work has shown the potential of DL-assisted pseudoaveraging in improving macular OCT image quality.21 However, the majority of these studies are limited by relatively small sample sizes and evaluation from a single camera and institution. There exists a gap in knowledge regarding the development of robust DL-based image quality assessment tools that are generalizable to other cameras and institutions.
Artificial Intelligence Ready and Exploratory Atlas for Diabetes Insights (AI-READI) is a multicenter data-generation project aiming to promote widespread use of AI approaches to study type 2 diabetes mellitus salutogenesis.22,23 This data set aims to enroll 4000 people aged 40 years or older, balanced by self-reported race and ethnicity, diabetes status, and reported biological sex23 and contains data including: vital signs, retinal imaging (fundus photography, OCT, and OCTA), electrocardiogram, cognitive function testing, continuous glucose monitoring, physical activity, home air quality, blood and urine collection for laboratory testing, and psychosocial variables including social determinants of health. The AI-READI data set version 2.0.0 enrolled 1067 participants between July 19, 2023, and July 31, 202424,25 and contains multiple disc- and macula-centered OCTA photographs at the superficial, deep, outer, and choriocapillaris plexuses for each enrolled patient.25 Due to the robust collection of OCTA photographs across institutions and cameras, we hypothesized that the AI-READI data set could be utilized to fine-tune DL models for generalizable assessment of OCTA image quality.
The purpose of our study was twofold: 1) to develop DL models to assess image quality of OCTA photographs and 2) to assess model generalizability on an internal test set of unseen photographs from AI-READI and an external test set of OCTA photographs acquired from a different institution and camera.
Methods
Training and Validation Data Set Development
Data collection and dissemination for AI-READI was previously approved by the University of Washington institutional review board #STUDY00016228). Patient data used from the Massachusetts Eye and Ear Infirmary (MEEI) data set was also previously approved by the Massachusetts General Brigham institutional review board (#2019P001863). Informed consent was obtained for all patients at MEEI. This study adheres to the Declaration of Helsinki.
All patients recruited to AI-READI were required to undergo multimodal data collection, including OCTA photographs from the ZEISS Cirrus 5000 (ZEISS), Topcon Maestro2 (Topcon), and Topcon Triton OCTA (Topcon) cameras. All 6 × 6-mm en face OCTA superficial plexus slabs from the version 2.0.0 AI-READI data set for all patients were included. We chose to analyze 6 × 6-mm commercial OCTA photographs at the superficial plexus due to its importance in retinal perfusion and importance of pathology occurring at this vascular level. There were no additional inclusion or exclusion criteria. All photographs used in this study are publicly available at the following link: https://fairhub.io/datasets/2. Demographic data including gender, race, and ethnicity were not included for patient privacy, although summary demographic data is available at the aforementioned link and in Table 1. Following labeling, the photographs were distributed into predefined training, validation, and testing sets balanced by AI-READI for demographic features and diabetes status.
Table 1.
Distribution of OCT Angiography Photographs and Available Patient Characteristics in the AI-READI and MEEI Data Sets
| AI-READI |
MEEI |
|||
|---|---|---|---|---|
| Training Split | Validation Set | Test Set (Both 6x6 and 12x12) | External Test Set | |
| Count (n [%]) | 4406 (70.3%) | 943 (15.0%) | 920 (14.7%) | 1100 |
| Total number of patients (n [%]) | 741 (70.0%) | 159 (15.0%) | 159 (15.0%) | 539 (100.0%) |
| Labels (n [%]) | ||||
| Good quality | 2842 (64.5%) | 593 (62.9%) | 563 (61.1%) | 895 (81.4%) |
| Bad quality | 1464 (35.5%) | 350 (37.1%) | 357 (38.9%) | 205 (18.6%) |
| Diabetes status (n [%]) | ||||
| Healthy | 1745 (39.7%) | 235 (24.9%) | 223 (24.2%) | 385 (71.4%) |
| Prediabetes | 960 (21.8%) | 243 (25.8%) | 234 (25.4%) | 76 (14.1%) |
| Diabetes controlled on oral medication | 1357 (30.8%) | 280 (29.7%) | 239 (26.0%) | 37 (6.8%) |
| Insulin-dependent diabetes | 344 (7.8%) | 185 (19.6%) | 224 (24.3%) | 41 (7.6%) |
| Age (mean ± standard deviation, years) | 60.6 ± 10.9 | 60.3 ± 11.3 | 60.2 ± 10.6 | 62.0 ± 17.7 |
AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; MEEI = Massachusetts Eye and Ear Infirmary.
External Data Set Development
A random subset of 1100 patients from MEEI with macula-centered 6x6 OCTA photographs of the superficial plexus taken by the ZEISS Plex Elite 9000 (ZEISS) were randomly sampled for an external test set. These photographs were taken during appointments for routine clinical care.
To assess model generalizability on photographs of different sizes, 12 × 12-mm OCTA photographs acquired by the Topcon Triton camera were also extracted from patients included in the testing set. Patients from the training and validation sets were excluded to eliminate model bias from learning patient-specific features.
Image Quality Labeling
Seven graders (two ophthalmologists [JSC, KM] and five medical students [LEW, AL, NR, LV JM]) graded all 6 × 6-mm OCTA en face superficial plexus slabs of the macula from the AI-READI data set as acceptable or poor quality based on the following criteria: 1) for the presence of banding, eye movement, vignetting, defocus, and shadow, and incorrect structure (i.e., optic nerve head) and 2) and if the cumulative presence of these artifacts obscured >15% of the photograph. Intergrader validation with 30 photographs was performed on a subsample to ensure similar grading by all graders, with a Fleiss kappa score of 0.71 between all graders demonstrating substantial agreement. Correlation constants between all anonymized graders are available in Supplemental Figure 1 (available at www.ophthalmologyscience.org). All graders reviewed at least 25% of the data set such that all images had multiple labels. Disagreements were resolved by discussion, and a final ground-truth label was determined using the mode.
For the two external test sets, an ophthalmologist (JSC) independently graded all photographs as acceptable or poor quality using similar criteria as above. Representative acceptable quality photos from the 6 × 6-mm AI-READI data set, MEEI data set, and 12x12 AI-READI test set are shown in Figure 1. Examples of poor-quality photographs and annotated artifacts are shown in Figure 2.
Figure 1.
Examples of acceptable quality 6 × 6-mm OCTA macula-centered photographs of the superficial vascular plexus from Topcon Triton (A), Topcon Maestro2 (B), and Zeiss Cirrus 56 000 (C) OCTA cameras in the AI-READI 6 × 6-mm data set, as well as from the PLEX ELITE (D), and 12 × 12-mm photographs from the Topcon Triton (E). AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; OCTA = OCT angiography.
Figure 2.
Examples of poor quality 6 × 6-mm OCTA macula-centered photographs of the superficial vascular plexus (A, B, C, D) with example annotations. Red, yellow, green, blue, orange, and purple arrows depict banding, eye movement, vignetting, defocus, and shadowing, respectively. Additionally, an example of the optic nerve head mislabeled as a macula-centered photograph is shown (E). OCTA = OCT angiography.
Deep Learning Methods
All photographs were preprocessed with Contrast-Limited Adaptive Histogram Equalization and resized to 224 × 224 pixels. Four state-of-the-art DL algorithms (ResNet-50, EfficientNet, vision transformers [ViT], and ConvNeXt_v2) were fine-tuned. All models were previously pre-trained on the ImageNet data set. Hyperparameter search for the optimizer (Stochastic Gradient Descent, Adam, and AdamW), learning rate, weight decay, dropout, and freezing of the model backbones or not was performed for training and validation over 100 epochs, with 10 epochs of no improvement set for early stopping. Transformations to the training photographs, including rotations, flips, and color jitter, were applied to each image once per epoch for image augmentation. For each algorithm, the fine-tuned model with the lowest validation loss was selected for testing. All model development was performed using Python version 3.9, PyTorch 2.5.1, and an Nvidia RTX A6000 GPU. More information regarding our training parameters can be found in Supplementary Document 2 (available at www.ophthalmologyscience.org). Learning curves for loss and accuracy for our best performing models are additionally provided for our models in Supplementary Figure 3 (available at www.ophthalmologyscience.org).
All models were then evaluated on test sets consisting of 6 × 6-mm and 12 × 12-mm photographs from AI-READI and the external 6 × 6-mm test set from MEEI. Performance was evaluated on accuracy, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) scores.
Explainability
To facilitate explainability, Gradient-weighted Class Activation Mapping (Grad-CAM) was used to assess specific image features the models focused on and generate heatmaps. Gradients were extracted from the final convolutional layer of the last residual or stage block for ResNet, EfficientNet, and ConvNeXt v2. For the ViT, Grad-CAM was used at the final LayerNorm layer of the encoder after a reshape transformation. These heatmaps were reviewed by JSC for thematic elements.
Statistical Analysis
Statistical analysis was performed using R version 4.3.0. The number of photographs and patients in the AI-READI training, validation, and test sets and from the MEEI test set were enumerated by proportion of acceptable and poor-quality image labels. Demographic data (reported gender and race/ethnicity) and diabetes status (healthy, prediabetes, diabetes controlled on oral medication or noninsulin injectable, and insulin-dependent diabetes) previously provided by AI-READI were summarized. The MEEI electronic health record was reviewed for similar data. The mean age and standard deviation were calculated for all patients in both data sets. The performance of all models on each test set was evaluated on accuracy, sensitivity, specificity, and AUROC score.
Results
Six thousand two hundred sixty-nine OCTA photographs from 1067 patients and 1100 OCTA photographs from 556 patients were included from the AI-READI and MEEI data sets, respectively. Additionally, 325 12 × 12-mm OCTA photographs from 160 patients from the AI-READI data set were included. The mean age was 60.3 ± 11.2 years in the AI-READI data set and 62.0 ± 17.7 years in the MEEI data set. Within the AI-READI data set, there were 2203 (35.1%) healthy patients, 1437 (22.9%) patients with prediabetes, 1876 (29.9%) patients with diabetes controlled on oral medication or noninsulin injectables, and 753 (12.0%) patients with diabetes on insulin and other injectables. Within the MEEI data set, there were 394 (70.7%) healthy patients, 82 (14.7%) patients with prediabetes, 37 (6.6%) patients with diabetes controlled on oral medication or noninsulin injectables, and 43 (7.6%) patients with diabetes on insulin and other injectables. Available demographic and data set characteristics and adapted demographic characteristics from the AI-READI summary table are shown in Table 1.
The accuracies and AUROCs of the fine-tuned ResNet, EfficientNet, ViT, and ConvNeXt v2 models on the AI-READI validation set were 82.5%, 82.9%, 83.7%, and 83.5%, and the AUROCs were 0.906, 0.904, 0.907, and 0.913, respectively. The accuracy of the fine-tuned ResNet, EfficientNet, ViT, and ConvNeXt v2 models on the internal test set from AI-READI were 83.5%, 82.5%, 83.5%, and 82.9%, and the AUROCs were 0.904, 0.904, 0.908, and 0.913, respectively. The corresponding sensitivities were 77.9%, 73.4%, 69.2%, and 79.8%, and specificities were 86.1%, 89.9%, 92.7%, and 84.2%, respectively. These data are shown in Figure 3.
Figure 3.
Area under the receiver operating characteristic curves for validation (A) and test (B) performance from deep learning models on the AI-READI 6 × 6-mm OCTA photograph data set. AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; OCTA = OCT angiography; AUROC = area under the receiver operating characteristic; ViT = Vision Transformer.
The models were also evaluated on an external test set of 6 × 6-mm superficial plexus OCTA photograph from MEEI and the 12 × 12-mm OCTA photographs test set from AI-READI. The accuracy and AUROC of the fine-tuned ResNet, EfficientNet, ViT, and ConvNeXt v2 models on the MEEI test set were 91.5%, 94.3%, 93.9%, and 88.0%, respectively. The AUROCs were 0.925, 0.974, 0.974, and 0.957, respectively. This corresponded to sensitivities of 73.2%, 84.5%, 86.8%, and 84.4% and specificities of 98.4%, 97.3%, 97.2%, and 92.7%, respectively. The accuracies for the 12 × 12-mm AI-READI test set were 72.3%, 76.4%, 66.2%, and 83.7%, corresponding to AUROCs of 0.775, 0.775, 0.760, and 0.834, respectively. The sensitivities were 95.7%, 95.7%, 97.9%, and 96.4%, respectively, and the specificities were 54.5%, 57.8%, 42.2%, and 74.1%, respectively. These data are shown in Figure 4.
Figure 4.
Area under the receiver operating characteristic curves for deep learning models evaluated on 6 × 6-mm en face OCTA superficial plexus slabs from the MEEI test set (A) and 12 × 12-mm test set from AI-READI (B). AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; OCTA = OCT angiography; AUROC = area under the receiver operating characteristic; MEEI = Massachusetts Eye and Ear Infirmary; ViT = Vision Transformer.
Gradient-weighted Class Activation Mapping heat maps for all 4 models are available in Supplemental Figure 4 (available at www.ophthalmologyscience.org). Overall, these heatmaps showed that all ResNet, EfficientNet, and ConvNeXt v2 models found that imaging features corresponding to image artifacts (banding, eye movement, vessel discontinuity, and various blurs) were important in determining image quality. Additionally, Grad-CAM showed that ViT focused largely on vessel-based features.
Discussion
In this study, we trained four state-of-the-art DL architectures to assess acceptable versus poor quality 6 × 6 en face OCTA images of the superficial plexus in the AI-READI data set and evaluated these images on unseen OCTA images in the AI-READI data set and an external data set from MEEI. Our study has the following two key findings: 1) DL models can be trained to assess image quality of OCTA photographs from AI-READI, and 2) image quality assessment of models trained on data from AI-READI is generalizable to photographs from other institutions, cameras, and image sizes.
The first key finding is that DL models can be trained to assess image quality of OCTA photographs from AI-READI. Overall, our models performed with accuracy and AUROC ranging from 73.7%–82.5% and 0.904–0.913, respectively, on the internal test set. Although the newest generation model ConvNeXt v2 demonstrated the highest performance, an important reason all models performed relatively similarly may be that the more recently developed architectures (ViT and ConvNeXt v2) perform most optimally with larger data sets consisting of millions of images, such as the benchmark imaging data set ImageNet.26 Additionally, Grad-CAM analysis demonstrated that models were generally able to recognize features for image quality analysis, including vessel-based and background features. While AI-READI is to date the largest publicly available collection of OCTA images, the data set is poised to continue growing with ongoing patient recruitment. In fact, AI-READI version 3.0.0 was recently released in November 2025, with complete multimodal data from 2280 participants.27 There are several potential applications of this model, including image reconstruction28,29 and disease diagnosis,30,31 image quality screening is a key step for optimizing data sets for model performance. Within clinical practice, automated image quality screening may be useful for live feedback for photographers obtaining OCTA scans. Similarly, assessing image quality for OCTAs obtained for retrospective studies and prospective clinical trials may be useful to optimize trial-related imaging metrics. An important limitation of this model is its focus on en face 6 × 6 mm of the superficial plexus; image quality assessment of OCTA photographs of the deep and outer plexuses as well as the choriocapillaris was not evaluated due to relatively high image artifact (likely largely related to signal attenuation) compared to photographs of the superficial plexus. These layers represent key opportunities for ongoing work in image quality assessment. Future work may focus on separate models evaluating image quality within en face view of each plexus or three-dimensional OCT photographs.
The second key finding is that image quality assessment of models trained on data from AI-READI is generalizable to photographs from other institutions, cameras, and image sizes. Despite training and validation performed on photographs acquired by three other cameras, models tested on OCTA photographs from the Zeiss PLEX Elite, which were not included as part of the training or validation process, performed with accuracy AUROCs ranging from 88.0%–94.9% and 0.925–0.974, respectively. The best performing model on the MEEI data set was the ViT, with a sensitivity of 86.8% and specificity of 97.2%. There are two potential reasons for the higher performance on the PLEX images compared to the following internal test set: 1) both cameras were manufactured by Zeiss and may have similar pixel and processing patterns, and 2) there were relatively higher proportions of high-quality images in this data set. These findings are consistent with prior work demonstrating improved DL model generalizability when data sets are integrated from multiple cameras and institutions.32,33 Additionally, we demonstrate moderate generalizability of our models to 12 × 12-mm OCTA photographs, with accuracies and AUROC ranging from 66.2%–83.7% and 0.760–0.834, respectively. One reason for the performance of these models on the 12 × 12-mm photographs is that the models may have learned imaging landmarks of the macula (i.e., fovea, boundaries of arcade vessels) in the 6 × 6-mm field of view while assessing image quality features, which differ in 12 × 12-mm photographs. Incorporation of attention, fine-tuning on images of different sizes, and scale-aware training may mitigate the impact of field-of-view on performance. This highlights the ongoing need for diverse data sets to improve generalizability. Beyond standard online transformations and routine hyperparameter search, we did not use performance boosting features such as oversampling, feature selection, and offline data augmentation before data splitting, which have been shown to boost internal data set performance at the expense of generalizability.34 This model has the potential for out-of-the-box use for other OCTA cameras, although assessment of model performance over time both on OCTA devices trained on in this study as well as unseen OCTA devices is warranted. Temporal domain shift due to population or device-related changes remains a key area of research in assessing prospective performance degradation in AI models.35,36 Ongoing research in domain adaptation methods is necessary to address generalizability of AI models affected by the dynamic nature of clinical care and research.35,37
This study highlights a few of the many potential key advantages and applications of the AI-READI data set. One key advantage of this data set is the high variability in image quality, including photographs ranging from minimal artifacts to severe image artifacts impacting significant portions of photographs. The range of image quality in this data set is likely reflective of prospective conditions for image acquisition and likely to be of benefit in increasing model generalization. Another advantage of the AI-READI data set is the truly multimodal nature of the data set, including other retinal photographs, OCT scans, laboratory data, surveys, cognitive assessments, monofilament testing, electrocardiogram, and wearable data, which may facilitate multimodal models for better understanding diabetes salutogenesis and oculomic studies.38 Specifically, due to the wealth of systemic data in this data set, it may be possible to study systemic associations with ocular biomarkers in context of diabetes within AI-READI. Third, data in AI-READI were created using Findable, Accessible, Interoperable, and Reusable principles, highlighting a potential roadmap for compilation of future open-source data sets which may be used to study other ophthalmic and systemic diseases.39,40
This study has additional limitations. First, the majority of these patients had healthy maculas. Future work should include image assessment with other retinal pathology. Second, binary image quality metrics (or even proprietary metrics such as signal strength) may oversimplify prospective assessment of image quality and clinical assessment of images. Specifically, clinical OCTA metrics such as vessel density and foveal avascular zone measurement may be calculable and accurate even with prevalent image artifacts elsewhere. Third, we only used deep learning–based methods in this study and did not evaluate vision language models in image quality assessment, including LLaVA and medGemma. More research is needed in evaluating other AI methods for image quality assessment, as well as how large language models may be fine-tuned for diagnosis, disease forecasting, among other ophthalmic applications.
Overall, the AI-READI data set can be used to train high-performing and generalizable DL models for automated OCTA image quality assessment. As OCTA technologies and clinical applications continue to improve, image quality assessment will become increasingly important in developing future AI models and improving clinical use and interpretation of images.
Manuscript no. XOPS-D-25-01134.
Footnotes
Supplemental material available atwww.ophthalmologyscience.org.
Disclosure(s):
The Article Publishing Charge (APC) for this article was paid by University of California San Diego.
All authors have completed and submitted the ICMJE disclosures form.
The author(s) have made the following disclosure(s):
N.A.P.: Consultant – 4dmt, Apellis, Alcon, Allergan, Biogen, DORC, Alimera, Eye Point, Genentech, Kyoto, Regenx Bio, Regeneron, YoungMD connect.
J.B.M.: Consultant – Alcon, Topcon, Zeiss, Sumitomo, Genentech.
S.L.B.: Consultant – Topcon, Alcon.
J.S.C. receives funding from the Heed Foundation Fellowship. L.E.W. was supported by the National Institutes of Health under award T32GM007198-48S1 (UC San Diego Medical Scientist Training Program). S.L.B. receives funding from National Institutes of Health (awards OT2OD032644 and P30EY022589).
HUMAN SUBJECTS:
Human subjects were included in this study. Data collection and dissemination for AI-READI were previously approved by the University of Washington institutional review board (IRB #STUDY00016228). The patient data used from the Massachusetts Eye and Ear Infirmary (MEEI) data set were also previously approved by the Massachusetts General Brigham IRB (2019P001863). Informed consent was obtained for all patients at MEEI. This study adheres to the Declaration of Helsinki.
No animal subjects were used in this study.
Author Contributions:
Conception and design: Chen, Baxter
Analysis and interpretation: Chen, Baxter
Data collection: Chen, Wedekind, Legala, Tenerelli, Most, Ramesh, Vo, Chang, Marra, Ding, Lin
Obtained funding: N/A
Overall responsibility: Chen, Hribar, Husain, Kim, Wang, Patel, Miller, Baxter
Supplementary Data
Inter-rater correlation matrix demonstrating correlation between graders for image quality.
Training conditions including more detailed explanations on model hyperparameters, PyTorch conditions, and for this study.
Learning curves evaluating loss and accuracy from the best performing models (ResNet, EfficientNet, ViT, and ConvNeXt V2) in this study. ViT = Vision Transformer.
Gradient-weighted Class Activation Mapping heatmaps for all models and data sets in this study. Grad-CAM and paired original OCTA photographs for all models and datasets in this study including AIREADI 6 × 6 (A), MEEI (B), and AIREADI 12 × 12 (C). AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; MEEI = Massachusetts Eye and Ear Infirmary; ViT = Vision Transformer; Grad-CAM = Gradient-weighted Class Activation Mapping.
References
- 1.Wei X., Camino A., Pi S., et al. Real-time cross-sectional and en face OCT angiography guiding high-quality scan acquisition. Opt Lett. 2019;44(6):1431–1434. doi: 10.1364/OL.44.001431. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ding X., Xu J., Romano F., et al. Association of OCT angiography-detected intraretinal microvascular abnormalities with diabetic retinopathy severity. Ophthalmol Retina. 2025;10:185–193. doi: 10.1016/j.oret.2025.08.011. [DOI] [PubMed] [Google Scholar]
- 3.Ryu G., Lee K., Park D., et al. A deep learning model for identifying diabetic retinopathy using optical coherence tomography angiography. Sci Rep. 2021;11(1) doi: 10.1038/s41598-021-02479-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Finn M., Baldwin G., Garg I., et al. Comparative study of widefield swept-source optical coherence tomography angiography in eyes with concomitant age-related macular degeneration and diabetic retinopathy. Br J Ophthalmol. 2024;108(7):963. doi: 10.1136/bjo-2023-323792. [DOI] [PubMed] [Google Scholar]
- 5.Xiong H., You Q.S., Guo Y., et al. Deep learning-based signal-independent assessment of macular avascular area on 6×6 mm optical coherence tomography angiogram in diabetic retinopathy: a comparison to instrument-embedded software. Br J Ophthalmol. 2023;107(1):84. doi: 10.1136/bjophthalmol-2020-318646. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Heinke A., Zhang H., Broniarek K., et al. Cross-instrument optical coherence tomography-angiography (OCTA)-based prediction of age-related macular degeneration (AMD) disease activity using artificial intelligence. Sci Rep. 2024;14(1) doi: 10.1038/s41598-024-78327-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ma J., Desai R., Nesper P., et al. Optical coherence tomographic angiography imaging in age-related macular degeneration. Ophthalmol Eye Dis. 2017;9 doi: 10.1177/1179172116686075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hoyek S., Chaaya C., Lemire C.A., et al. Retinal imaging biomarkers and correlation to systemic disease activity in pediatric sickle cell disease. Ophthalmol Sci. 2025;5(5) doi: 10.1016/j.xops.2025.100774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Alam M., Thapa D., Lim J.I., et al. Quantitative characteristics of sickle cell retinopathy in optical coherence tomography angiography. Biomed Opt Express, BOE. 2017;8(3):1741–1753. doi: 10.1364/BOE.8.001741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Samara W.A., Shahlaee A., Sridhar J., et al. Quantitative optical coherence tomography angiography features and visual function in eyes with branch retinal vein occlusion. Am J Ophthalmol. 2016;166:76–83. doi: 10.1016/j.ajo.2016.03.033. [DOI] [PubMed] [Google Scholar]
- 11.Battaglia Parodi M., Arrigo A., Antropoli A., et al. Deep capillary plexus as biomarker of peripheral capillary nonperfusion in central retinal vein occlusion. Ophthalmol Sci. 2023;3(2) doi: 10.1016/j.xops.2022.100267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cheung C.M.G., Yanagi Y., Akiba M., et al. Improved detection and diagnosis of polypoidal choroidal vasculopathy using a combination of optical coherence tomography and optical coherence tomography angiography. Retina. 2019;39(9):1655–1663. doi: 10.1097/IAE.0000000000002228. [DOI] [PubMed] [Google Scholar]
- 13.Costanzo E., Cohen S.Y., Miere A., et al. Optical coherence tomography angiography in central serous chorioretinopathy. J Ophthalmol. 2015;2015(1) doi: 10.1155/2015/134783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wu J.H., Moghimi S., Nishida T., et al. Detection and agreement of event-based OCT and OCTA analysis for glaucoma progression. Eye. 2024;38(5):973–979. doi: 10.1038/s41433-023-02817-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kamalipour A., Moghimi S., Khosravi P., et al. Combining optical coherence tomography and optical coherence tomography angiography longitudinal data for the detection of visual field progression in glaucoma. Am J Ophthalmol. 2023;246:141–154. doi: 10.1016/j.ajo.2022.10.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dhodapkar R.M., Li E., Nwanyanwu K., et al. Deep learning for quality assessment of optical coherence tomography angiography images. Sci Rep. 2022;12(1) doi: 10.1038/s41598-022-17709-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lauermann J.L., Treder M., Alnawaiseh M., et al. Automated OCT angiography image quality assessment using a deep learning algorithm. Graefe’s Archive Clin Exp Ophthalmol. 2019;257(8):1641–1648. doi: 10.1007/s00417-019-04338-7. [DOI] [PubMed] [Google Scholar]
- 18.Jin Y., Gui F., Chen M., et al. Deep learning-driven automated quality assessment of ultra-widefield optical coherence tomography angiography images for diabetic retinopathy. Vis Computer. 2025;41(2):1049–1059. doi: 10.1007/s00371-024-03383-6. [DOI] [Google Scholar]
- 19.Lee T., Jammal A.A., Mariottoni E.B., Medeiros F.A. Predicting Glaucoma development with longitudinal deep learning predictions from Fundus photographs. Am J Ophthalmol. 2021;225:86–94. doi: 10.1016/j.ajo.2020.12.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kamalipour A., Moghimi S., Hou H., et al. OCT angiography artifacts in glaucoma. Ophthalmology. 2021;128(10):1426–1437. doi: 10.1016/j.ophtha.2021.03.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Abu-Qamar O., Lewis W., Mendonca L.S.M., et al. Pseudoaveraging for denoising of OCT angiography: a deep learning approach for image quality enhancement in healthy and diabetic eyes. Int J Retina Vitreous. 2023;9(1):62. doi: 10.1186/s40942-023-00486-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Baxter S.L., de Sa V.R., Ferryman K., et al. AI-READI: rethinking AI data collection, preparation and sharing in diabetes research and beyond. Nat Metab. 2024;6(12):2210–2212. doi: 10.1038/s42255-024-01165-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Owsley C., Matthies D.S., McGwin G., et al. Cross-sectional design and protocol for Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights (AI-READI) BMJ Open. 2025;15(2) doi: 10.1136/bmjopen-2024-097449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Flagship Dataset of Type 2 diabetes from the AI-READI Project. https://fairhub.io/datasets/2
- 25.Hub F.D.I. Documentation for the AI-READI dataset. https://docs.aireadi.org/
- 26.Deng J., Dong W., Socher R., et al. 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; New York City, NY: 2009. ImageNet: a large-scale hierarchical image database; pp. 248–255. [DOI] [Google Scholar]
- 27.Flagship Dataset of Type 2 diabetes from the AI-READI Project. https://fairhub.io/datasets/3
- 28.Le D., Son T., Kim T.H., et al. Deep learning-based optical coherence tomography angiography image construction using spatial vascular connectivity network. Commun Eng. 2024;3(1):28. doi: 10.1038/s44172-024-00173-9. [DOI] [Google Scholar]
- 29.Gao M., Guo Y., Hormel T.T., et al. Reconstruction of high-resolution 6×6-mm OCT angiograms using deep learning. Biomed Opt Express. 2020;11(7):3585–3600. doi: 10.1364/BOE.394301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Bowd C., Belghith A., Zangwill L.M., et al. Deep learning image analysis of optical coherence tomography angiography measured vessel density improves classification of healthy and glaucoma eyes. Am J Ophthalmol. 2022;236:298–308. doi: 10.1016/j.ajo.2021.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Zang P., Hormel T.T., Hwang T.S., et al. Deep-learning-aided diagnosis of diabetic retinopathy, age-related macular degeneration, and glaucoma based on structural and angiographic OCT. Ophthalmol Sci. 2023;3(1) doi: 10.1016/j.xops.2022.100245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Chen J.S., Coyner A.S., Ostmo S., Sonmez K., Bajimaya S., Pradhan E., Valikodath N., Cole E.D., Al-Khaled T., Chan R.V.P., Singh P., Kalpathy-Cramer J., Chiang M.F., Campbell J.P. Deep learning for the diagnosis of stage in retinopathy of prematurity: accuracy and generalizability across populations and cameras. Ophthalmol Retina. 2021;5:1027–1035. doi: 10.1016/j.oret.2020.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rockenschaub P., Hilbert A., Kossen T., et al. The impact of multi-institution datasets on the generalizability of machine learning prediction models in the ICU. Crit Care Med. 2024;52(11):1710–1721. doi: 10.1097/CCM.0000000000006359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Maleki F., Ovens K., Gupta R., et al. Generalizability of machine learning models: quantitative evaluation of three methodological pitfalls. Radiol Artif Intelligence. 2023;5(1) doi: 10.1148/ryai.220028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Stacke K., Eilertsen G., Unger J., Lundström C. Measuring domain shift for deep learning in histopathology. IEEE J Biomed Health Inform. 2021;25(2):325–336. doi: 10.1109/JBHI.2020.3032060. [DOI] [PubMed] [Google Scholar]
- 36.Pooch E.H.P., Ballester P.L., Barros R.C. Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classification. 2019. https://arxiv.org/abs/1909.01940v1
- 37.Ren J., Hacihaliloglu I., Singer E.A., et al. Unsupervised domain adaptation for classification of histopathology whole-slide images. Front Bioeng Biotechnol. 2019;7:2019. doi: 10.3389/fbioe.2019.00102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Weinreb R.N., Lee A.Y., Baxter S.L., et al. Application of artificial intelligence to deliver healthcare from the eye. JAMA Ophthalmol. 2025;143:529–535. doi: 10.1001/jamaophthalmol.2025.0881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Gim N., Shaffer J., Owen J., et al. Streamlined DICOM standardization in retinal imaging: bridging gaps in ophthalmic healthcare and AI research. Invest Ophthalmol Vis Sci. 2024;65(7):5879. [Google Scholar]
- 40.Matthies D., Owen J., McGwin G., et al. Generation of a multimodal Atlas of Type 2 Diabetes for Artificial Intelligence (AI-READI): Purpose and design. Invest Ophthalmol Vis Sci. 2024;65(7):3728. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Inter-rater correlation matrix demonstrating correlation between graders for image quality.
Training conditions including more detailed explanations on model hyperparameters, PyTorch conditions, and for this study.
Learning curves evaluating loss and accuracy from the best performing models (ResNet, EfficientNet, ViT, and ConvNeXt V2) in this study. ViT = Vision Transformer.
Gradient-weighted Class Activation Mapping heatmaps for all models and data sets in this study. Grad-CAM and paired original OCTA photographs for all models and datasets in this study including AIREADI 6 × 6 (A), MEEI (B), and AIREADI 12 × 12 (C). AI-READI = Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; MEEI = Massachusetts Eye and Ear Infirmary; ViT = Vision Transformer; Grad-CAM = Gradient-weighted Class Activation Mapping.