Multimodal deep transfer learning to predict retinal vein occlusion macular edema recurrence after anti-VEGF therapy

Abstract

Purpose

To develop a multimodal deep transfer learning (DTL) fusion model using optical coherence tomography angiography (OCTA) images to predict the recurrence of retinal vein occlusion (RVO) and macular edema (ME) after three consecutive anti-VEGF therapies.

Methods

This retrospective cross-sectional study consisted of 2800 B-scan OCTA macular images collected from 140 patients with RVO-ME. The central macular thickness (CMT) > 250 μm was used as a criterion for recurrence in the three-month follow-up after three injections of anti-VEGF therapy. The qualified OCTA image preprocessing and the lesion area segmentation were performed by senior ophthalmologists. We developed and validated the clinical, DTL, and multimodal fusion models based on clinical and extracted OCTA imaging features. The performance of the models and experts predictions were evaluated using several performance metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

Results

The DTL models exhibited higher prediction efficacy than the clinical models and experts’ predictions. Among the DTL models, the Vgg19 performed better than that of the other models, with an AUC of 0.968 (95 % CI, 0.943–0.994), accuracy of 0.913, sensitivity of 0.922, and specificity of 0.902 in the validation cohort. Moreover, the fusion Vgg19 model showed the highest prediction efficacy among all the models, with an AUC of 0.972 (95 % CI, 0.946–0.997), accuracy of 0.935, sensitivity of 0.935, and specificity of 0.934 in the validation cohort.

Conclusions

Multimodal fusion DTL models showed robust performance in predicting RVO-ME recurrence and may be applied to assist clinicians in determining patients’ follow-up time after anti-VEGF therapy.

1. Introduction

Retinal vein occlusion (RVO) complicated by macular edema (ME) is an ophthalmological disease that poses a grim threat to human vision [1]. The disruption of the vascular barrier and impaired return of tissue fluid return secondary to vascular occlusion are the main primary mechanisms involved in its pathogenesis; however, these mechanisms are still not fully understood [2]. Effective control of ME or prevention of post-treatment recurrence is important for preserving vision.

Anti-VEGF therapy can facilitate the absorption of retinal fluid and suppress neovascular leakage. It is currently the first-line therapy for managing ME in RVO cases [3]. The efficacy of the mainstream treatment strategies, such as pro re nata (PRN) or treat-and-extend (T&E), depends on the subjective judgment of macular central fovea edema [4]. It is often challenging for clinicians to predict the efficacy of treatment and determine the follow-up date before treatment because some patients respond poorly to anti-VEGF therapy or experience recurrence after improvement.

Optical coherence tomography angiography (OCTA), which quantifies macular foveal thickness, has been broadly used to evaluate the efficacy of treatment and recurrence of ME [5]. However, frequent follow-ups and examinations increase the burden on patients and strain healthcare resources [6]. It is difficult for ophthalmologists to predict the recurrence of RVO-ME after treatment using OCTA images. Therefore, there is an urgent need to develop a reliable prediction approach.

Deep learning is rapidly advancing in artificial intelligence. Convolutional neural networks (CNNs) are representative algorithms for extracting medical image features and are widely used to analyze conditions, such as glaucoma, high myopia, and retinal vascular diseases [7,8]. CNNs directly process images, automatically extracting features and conducting classification [9]. Their ability to extract features from OCTA images provides a promising solution for predicting disease progression and determining treatment effects [10]. Feng et al. used a CNN-based transfer learning to automatically predict the effectiveness of anti-VEGF therapy using OCT images before treatment, with a prediction area under the receiver operating characteristic (ROC) curve (AUC) exceeding 0.8 [11]. To the best of our knowledge, few studies have used artificial intelligence to predict the response to anti-VEGF therapy from baseline OCT in patients with RVO. Xu et al. used a generative adversarial network to predict the short-term response of patients with RVO to anti-VEGF therapy [12]. We enrolled patients who underwent treatment with three consecutive anti-VEGF injections over a 3 months follow-up period to confirm the relative long-term efficacy of anti-VEGF for RVO-ME. Methodologically, we constructed fusion models by combining deep transfer learning (DTL) and clinical features to predict the recurrence of RVO-ME using OCTA images. We aimed to develop a multimodal DTL fusion model based on OCTA images to predict whether RVO or ME will recur after three consecutive anti-VEGF therapies.

The main contributions of our study are as follows: (1) We explored the optimal method to use segmentation results provided by retinal physicians for CNNs training. (2) We developed CNN-based methods that can automate ME recurrence in OCTA images, achieving compelling performance in comparison to assessments by three physicians. (3) We combined DTL and clinical features to construct fusion models that achieved better performance. (4) These models guide subsequent treatment and follow-up, potentially serving as imaging markers to determine the recurrence of ME in RVO.

2. Materials and methods

2.1. Study population

The patient flowchart for this study is shown in Fig. 1. This retrospective cross-sectional study included 140 patients. Patients aged ≥18 years with central macular thickness (CMT) ≤ 250 μm of the first review of examination within 1 week after three consecutive anti-VEGF injections at Nanjing Medical University Affiliated Eye Hospital between February 2018 and June 2022 were included in the study. We excluded patients with the following criteria: (1) patients with other fundus diseases, including age-related macular degeneration, diabetic retinopathy, glaucoma, uveitis, and ocular ischemic syndrome; (2) patients with previous intraocular surgery or intraocular dexamethasone injection; (3) severe cataract and vitreous hemorrhage combination, resulting in refractive media turbidity, for which the recurrence of patient was unclear; and (4) poor-quality OCTA images. The CMT> 250 μm within 3 months after three injections of anti-VEGF therapy was used as the criterion for ME recurrence. The Medical Ethics Committee approved this study, which adhered to the principles of the Declaration of Helsinki.

Fig. 1.

Patient flowchart for this study.

OCTA, optical coherence tomography angiography; FFA, fluorescein fundus angiography; PPV, pars plana vitrectomy.

2.2. OCTA image acquisition

The AngioVue software (Optovue RTVue XR Avanti, version: A2017,1,0,151; Optovue Inc. Fremont, CA) was used to perform macular central foveal scans measuring 3 mm × 3 mm or 6 mm × 6 mm [13]. Twenty B-scan images of the ME lesions were manually selected for each patient's baseline session, resulting in a total of 2800 images. CMT, foveal avascular zone, foveal avascular zone perimeter (PERIM), foveal density, and vascular density of the deep capillary plexus (whole, fovea, and parafovea) and superficial capillary plexus (whole, fovea, and parafovea) were obtained as clinical variables at the end of the scan using the system software measurement function. For subsequent feature extraction and model construction, all scanned images were saved in TIFF format to establish a standardized image format [14].

2.3. Clinical model construction

The clinical characteristics of the 140 patients were selected using univariate analyses. Features with high collinearity (correlation coefficient >0.9) were excluded from the analysis. The remaining clinical features were used to build the clinical models using logistic regression (LR), support vector machine (SVM), Random Forest (RF), Decision Tree (DT), and an ensemble method.

2.4. Image preprocessing and segmentation

The original OCTA images were preprocessed to eliminate background noise, including the normalization of input images. Each OCTA image was normalized to a range of 0–255 pixels and cropped according to the smoothed pixel intensity [15]. Manual segmentation of the preprocessed images, including all the B-scans within each OCT volume, was performed by two senior ophthalmologists. The segmentation results were evaluated by a single retinal specialist.

2.5. DTL model construction

Fig. 2 shows the artificial intelligence workflow and study flowchart. After pre-training on ImageNet for the Vgg19, resnet50, GoogLeNet, and Inception-v3 DTL models, DTL features were extracted using the penultimate layer. In total, 4096, 2048,1024, and 2048 features were extracted from the Vgg19, resnet50, GoogLeNet, and Inception-v3 DTL models, respectively [16,17]. These features were normalized and selected using Z-score and least absolute shrinkage and selection operator (LASSO) regression. The selected DTL features were evaluated based on accuracy and receiver operating characteristic (ROC) curves using four different classifiers: SVM; LR; RF; and DT [18]. To overcome the instability of a small sample size, we integrated the four benchmark models using an average ensemble of four classifiers. In addition, DTL and clinical features were combined to construct four fusion models and outputted using different machine learning classifiers (SVM, LR, RF, DT, and Ensemble) [15]. Deep learning features were selected, and the models were constructed using the “PyTorch” Python package (version 1.8.1). All the DTL features extraction and models’ construction were performed on a single NVIDIA Quadro RTX 3070Ti 16 GB GPU in Windows 10.

Fig. 2.

Artificial intelligence workflow and study flowchart.

OCTA, optical coherence tomography angiography; DTL, deep transfer learning; SVM, support vector machine; LR, logistic regression; RF, Random Forest; DT, Decision Tree; CMT, central macular thickness, FAZ, foveal avascular zone, PERIM, FAZ perimeter, FD, foveal density, SCP, superficial capillary plexus, DCP, deep capillary plexus.

2.6. Evaluation of predictive efficacy

The evaluation metrics of different models’ prediction efficacy were calculated, including the accuracy, area under the ROC curve (AUC), 95 % Confidence interval (CI), sensitivity, specificity, recall, and F1-score [19]. We also evaluated the accuracy of the clinicians (ZLH, 5 years of experience; XXZ, 11 years of work experience; and YJ, 21 years of experience) in predicting recurrence after anti-VEGF injections.

Accuracy serves as an essential indicator for evaluating classification models. It is defined as the ratio of the correct predictions to the total. Precision measures the number of true positives among all predicted positive data. Recall quantifies the percentage of correctly identified positive samples out of all positive samples. The F-Measure, also known as F-Score, is a statistic representing the weighted harmonic mean of precision and recall. It is frequently used to assess the quality of classification models. F = (a² + 1) * precision * recall/(a² * precision + recall). When a = 1, it becomes the commonly used F1 score: F1 = 2 * precision * recall/(precision + recall).

The AUC was defined as the area enclosed by the coordinate axis under the ROC curve. The abscissa of the ROC curve represents the false positive rate, while the ordinate represents the true positive rate. As the ROC curve alone may not clearly indicate which classifier is superior, the AUC value is often employed as the evaluation criterion. The larger AUC value signifies a better classifier.

2.7. Statistical analysis

Chi-square tests or Fisher's tests were used for categorical variables, and t-test or Mann–Whitney test were used for quantitative variables. The Shapiro–Wilk test was used to assess the normality of the data. The mean ± standard deviation or median with an interquartile range (IQR) were used for continuous variables. The Spearman rank correlation test was used to evaluate the correlation and was performed using Python (version 3.11.2, https://www.python.org/). Statistical significance was set at p < 0.05. significant. Statistical analyses were performed using SPSS (version 28.0) and R software (version 4.1.3, https://www.r-project.org/).

3. Results

3.1. Base characteristics

Patients diagnosed with RVO-ME were selected between February 2018 and June 2022 at the Affiliated Eye Hospital of Nanjing Medical University. The training cohort included a total of 112 participants (2240 macular B-scan images), including 49 patients without recurrence (25 [22.3 %] women; mean age, 58.71 years) and 63 patients with recurrence (31 [27.7 %] women; mean age, 54.65 years). A total of 28 cases (560 macular B-scan images) were included in the validation cohort: 12 cases without recurrence (six [21.4 %] women; mean age, 55.58 years) and 16 cases with recurrence (seven [25.0 %] women; mean age, 58.25 years). There were significant differences in the deep capillary plexus (whole and parafovea) in the training cohort. Baseline patient data are shown in Table 1.

Table 1.

Basic characteristics and clinical data of the study subjects.

Variable	Training cohort (n = 112)			Validation cohort (n = 28)
	Recurrence (n = 63)	Non-recurrence (n = 49)	P value	Recurrence (n = 16)	Non-recurrence (n = 12)	P value
Age, years	54.65 ± 12.01	58.71 ± 12.08	0.079	58.25 ± 10.53	55.58 ± 15.75	0.325
Gender			0.849			1
male	32 (28.6 %)	24 (21.4 %)		9 (32.1 %)	6 (21.4 %)
female	31 (27.7 %)	25 (22.3 %)		7 (25.0 %)	6 (21.4 %)
CMT(μm)	524 (371, 650.5)	526 (427, 593)	0.869	531.91 ± 181.77	587.75 ± 161.05	0.406
FAZ(mm2)	0.33 (0.25, 0.39)	0.32 (0.21, 0.37)	0.546	0.32 (0.28, 0.38)	0.30 (0.23, 0.32)	0.201
PERIM(mm)	2.57 (2.12, 2.81)	2.45 (1.95, 2.74)	0.428	2.57 (2.25, 2.97)	2.37 (2.07, 2.49)	0.171
FD(%)	45.4 (43.7, 48.2)	45.7 (43.2, 48.5)	0.801	46.14 ± 4.13	45.03 ± 2.62	0.422
SCP(%)
whole	40.7 (38.2, 43.4)	39.7 (37.4, 41.9)	0.219	41.22 ± 4.10	38.80 ± 3.41	0.353
fovea	23.5 (20.2, 27.6)	24.2 (19.2, 26.3)	0.935	27.46 ± 7.18	25.40 ± 6.03	0.599
parafovea	42.4 ± 3.99	41.24 ± 4.99	0.176	42.69 ± 4.33	40.23 ± 4.33	0.235
DCP(%)
whole	43.4 (40.9, 46.2)	41.1 (37.3, 43.4)	0.006	42.4 (38.8, 46.3)	42.7 (41.1, 44.1)	1
fovea	33.53 ± 10.24	32.03 ± 9.37	0.427	34.4 (26.0, 38.4)	34.4 (31.8, 41.7)	0.745
parafovea	44.5 (43.0, 47.2)	42.1 (39.1, 44.4)	0.007	42.7 (39.9, 47.6)	43.3 (42.1, 46.1)	0.727

CMT, central macular thickness; FAZ, foveal avascular zone; PERIM, FAZ perimeter; FD, foveal density; SCP, superficial capillary plexus; DCP, deep capillary plexus.

3.2. Clinical model construction

The results of the prediction of whether recurrence occurred within 3 months after anti-VEGF treatment in RVO patients using different classifiers (SVM, LR, DT, RF, and Ensemble) are shown in Table 2. The AUCs of the validation cohort for all the four classifiers were low. The four classifiers were further integrated to optimize their characteristics and overcome the instability caused by an inadequate sample size. The predictive results obtained using the different classifiers are shown in Fig. S1. The accuracy, AUC, sensitivity, specificity, precision, recall, and F1-score of ensemble were 0.964, 0.995 (95 % CI, 0.989–1.000), 0.936, 1.000, 1.000, 0.937, and 0.967 in the training cohort. And the accuracy, AUC, sensitivity, specificity, precision, recall, and F1-score were 0.714, 0.635 (95 % CI, 0.406–0.865), 0.750, 0.727, 0.750, 0.750, and 0.750 in the validation cohort (Fig. 3A, Table 2). In addition, we evaluated the accuracy of the three clinicians in predicting recurrence or non-recurrence after anti-VEGF treatment using 140 patient images from the dataset. The comprehensive results of the expert predictions are presented in Table S1.

Table 2.

Clinical models performance.

Clinical Models	AUC (95%CI)	Accuracy	Sensitivity	Specificity	Precision	Recall	F1-score
SVM
Training	0.851 (0.777–0.924)	0.821	0.794	0.857	0.877	0.794	0.833
Validation	0.578 (0.590–0.798)	0.643	0.563	0.750	0.750	0.563	0.643
LR
Training	0.639 (0.535–0.744)	0.661	0.587	0.771	0.755	0.587	0.661
Validation	0.693 (0.494–0.892)	0.679	0.500	0.917	0.889	0.500	0.640
DT
Training	0.997 (0.993–1.000)	0.973	0.984	0.959	0.969	0.984	0.976
Validation	0.480 (0.244–0.710)	0.642	1.000	0.167	0.615	1.000	0.762
RF
Training	0.954 (0.918–0.988)	0.928	0.167	0.959	0.966	0.905	0.934
Validation	0.453 (0.204–0.702)	0.643	0.938	0.273	0.625	0.938	0.750
Ensemble
Training	0.995 (0.989–1.000)	0.964	0.936	1.000	1.000	0.937	0.967
Validation	0.635 (0.406–0.865)	0.714	0.750	0.727	0.750	0.750	0.750

AUC, area under the curve; 95%CI, 95 % confidential interval. SVM, support vector machines; LR, Logistic Regression; DT, Decision Tree; RF, Random Forest.

Fig. 3.

Comparison of prediction efficacy in different models.

(A) ROC curve and histogram for prediction of clinical model. (B) ROC curve and histogram for prediction of DTL-Vgg19 model. (C) ROC curve and histogram for prediction of fusion-Vgg19 model. DTL, deep transfer learning; ROC, receiver operating characteristic; AUC, area under the curve.

3.3. DTL model construction

Owing to the poor prediction results of the clinical models and expert predictions, we extracted deep visual features from the OCTA images for deep learning. We selected four deep learning models (Vgg19, resnet50, GoogLeNet, and Inception-v3) to extract the features and input them into different classifiers (SVM, LR, DT, RF, and Ensemble). In a comparison of all classifiers, the ensemble performed better than the four classifiers. Therefore, the ensemble prediction results were selected as the final outputs. Among the four DTL models, Vgg19 showed the best predictive performance, with a training cohort accuracy of 0.984, AUC of 0.999 (95 % CI, 0.998–1.000), sensitivity of 0.990, and specificity of 0.975, precision of 0.981, recall of 0.990, and F1-score of 0.985. The accuracy, AUC, sensitivity, specificity, precision, recall, and F1-score were 0.913, 0.968 (95 % CI, 0.943–0.994), 0.922, 0.902, 0.922, 0.922, and 0.922 in the validation cohort (Fig. 3B). The prediction results for the patients with and without recurrence are presented in Table 3. Therefore, Vgg19 is more suitable for predicting ME recurrence within 3 months after anti-VEGF treatment in patients with RVO. Furthermore, all DTL models were more accurate than clinical models in predicting recurrence or non-recurrence, with AUCs >0.80. The details of the prediction results of the four DTL models are listed in Table 3.

Table 3.

DTL models performance.

DTL Models	AUC (95%CI)	Accuracy	Sensitivity	Specificity	Precision	Recall	F1-score
Vgg19
Training	0.999 (0.998–1.000)	0.984	0.990	0.975	0.981	0.990	0.985
Validation	0.968 (0.943–0.994)	0.913	0.922	0.902	0.922	0.922	0.922
resnet50
Training	0.988 (0.982–0.994)	0.944	0.971	0.910	0.931	0.971	0.951
Validation	0.888 (0.830–0.946)	0.833	0.923	0.717	0.809	0.923	0.862
GoogLeNet
Training	0.993 (0.985–1.000)	0.975	0.980	0.967	0.974	0.980	0.977
Validation	0.912 (0.866–0.958)	0.819	0.883	0.738	0.810	0.883	0.845
Inception-v3
Training	0.976 (0.964–0.989)	0.935	0.951	0.914	0.933	0.951	0.942
Validation	0.835 (0.769–0.901)	0.761	0.792	0.721	0.782	0.792	0.787

DTL, Deep transfer learning; AUC, area under the curve; 95%CI, 95 % confidential interval.

3.4. Fusion model performance

To improve the predictive efficacy of the DTL models, we combined the extracted deep visual features with clinical variables to construct a fusion model to predict RVO-ME recurrence 3 months after anti-VEGF treatment. As shown in Table 4, the Vgg19 fusion model exhibited appealing prediction performance. The accuracy, AUC, sensitivity, specificity, precision, recall, and F1-score were 0.985, 0.999 (95 % CI, 0.999–1.000), 0.993, 0.975, 0.981, 0.993, and 0.987 in the training cohort. And the accuracy, AUC, sensitivity, specificity, precision, recall, and F1-score were 0.935, 0.972 (95 % CI, 0.946–0.997), 0.935, 0.934, 0.947, 0.935 and 0.941 in the validation cohort (Fig. 3C). The performance results for predicting the recurrence and non-recurrence of RVO-ME are presented in Table 4. In comparison with the Vgg19 DTL model, the accuracy of the fusion Vgg19 model was elevated, indicating that the fusion model had a higher prediction accuracy. In addition, all fusion models had higher prediction accuracies for both recurrence and non-recurrence, with AUCs >0.85, as shown in Table 4.

Table 4.

Fusion models performance.

Fusion Models	AUC (95%CI)	Accuracy	Sensitivity	Specificity	Precision	Recall	F1-score
Vgg19
Training	0.999 (0.999–1.000)	0.985	0.993	0.975	0.981	0.993	0.987
Validation	0.972 (0.946–0.997)	0.935	0.935	0.934	0.947	0.935	0.941
resnet50
Training	0.995 (0.988–1.000)	0.978	0.987	0.967	0.974	0.987	0.981
Validation	0.923 (0.875–0.971)	0.891	0.922	0.852	0.888	0.922	0.904
GoogLeNet
Training	0.997 (0.995–1.000)	0.987	0.987	0.988	0.990	0.987	0.989
Validation	0.947 (0.914–0.980)	0.861	0.908	0.803	0.852	0.908	0.879
Inception-v3
Training	0.988 (0.982–0.994)	0.942	0.961	0.918	0.936	0.961	0.948
Validation	0.883 (0.826–0.940)	0.790	0.883	0.672	0.773	0.883	0.824

AUC, area under the curve; 95%CI, 95 % confidential interval.

3.5. Model visualization

To visualize the pivotal elements within OCTA images strongly associated with the prognosis of treatment outcomes 3 months after anti-VEGF treatment in patients with RVO, an admired occlusion test was performed to illustrate the results and augment model transparency. Employing a blank 100 × 100-pixel box, we systematically traversed every feasible spot in the image, and the credibility of the prediction was itemized. The highest point on the probability scale epitomizes the part of the OCTA image that is the most critical for convincing classification (shown in blue in Fig. 4). Furthermore, retinal specialists (ZLH, HY, and CJQ) confirmed if the regions recognized by the occlusion test indeed represented the most clinically distinctive predictive zones in RVO-ME affected eyes.

Fig. 4.

Models visualization.

The heatmap illuminates the zones of pathology in the RVO-ME. The blue and red parts represent high and low affiliations, respectively. RVO, retinal vein occlusion; ME, macular edema.

4. Discussion

RVO-ME is generally responsive to anti-VEGF therapy in most patients [20,21]; however, recurrent RVO-ME has been a very challenging issue for clinicians because it can lead to irreversible vision loss if not followed up and managed promptly [[21], [22], [23]]. Therefore, it is necessary to predict the recurrence of RVO-ME. For this purpose, four different DTL neural networks, Vgg19, resnet50, GoogLeNet, and Inception-v3, were applied to extract OCT image features and further fuse them with clinical features to optimize the results, which were inputted into different deep learning classifiers (SVM, LR, RF, DT, and Ensemble) in our research. The optimal performance of the fusion classifier resulted in an AUC of 0.972 (0.946–0.998), and an accuracy rate of 93.5 %. These results significantly outperformed the predictions made by ophthalmologists in the validation cohort. This reliable prediction system is valuable in guiding the clinical anti-VEGF treatment interval and setting the follow-up date. The image features extracted by DTL may be markers for predicting recurrence [24,25].

The pro re nata and treat-and-extend anti-VEGF protocol for the control of RVO complicated by ME is promising, according to the results of recent multicenter clinical trials, but requires multiple injections of anti-VEGF and monthly follow-up to assess treatment efficacy [26]. The number of injections and follow-up dates are flexible depending on the actual situation, which relies on the subjective judgment of the physician [27]. Arrigo et al. found that patients with BRVO-ME require 9.80 ± 5.39 doses of anti-VEGF, whereas patients who skip visits frequently lose the ideal opportunity for therapy [28]. Therefore, predicting whether RVO-ME will recur in the short term has significance in determining the timing of anti-VEGF injection and the follow-up interval. The use of artificial intelligence technology for the diagnosis and prognostic prediction of retinal diseases has become widespread, and the diagnostic ability of CNNs based on ophthalmic image features has reached the expert levels in some aspects [15,29,30]. Using OCT images, pixel-level features extracted using deep learning algorithms can assist in the diagnosis and treatment in the area of fundus diseases [31]. Rasti R et al. used pre-treatment OCT scans to predict the efficacy of anti-VEGF therapy in diabetic ME [32]; however, few studies have utilized CNN models to predict RVO-ME recurrence following anti-VEGF treatment. In this study, the characteristics from high-resolution OCTA images were extracted using pre-trained CNNs, and their analysis using algorithmic models demonstrated the potential to predict whether the edema will return within 3 months after anti-VEGF therapy. We used fusion models to further optimize the results and improve the prediction accuracy. Similarly, Liu B et al. automated the segmentation and characterization of macular disease's photoreceptor alterations by integrating four distinct CNN algorithms; and the fusion models outperformed each other with higher Dice coefficients, sensitivity, and specificity [15]. Our study presented an automated, objective, and effective image-based diagnostic technique that offers new perspectives for the clinical management of RVO-ME, including cost savings and improved efficiency. In addition, telemedicine has great potential for application in areas where resources are insufficient and trained human diagnosticians are scarce [33].

Predicting whether ME will recur after anti-VEGF treatment, personalized injection regimens, and follow-up dates is a step toward precision medicine. Such prediction may help clinicians better select first-line treatments for their patients [34,35]. Our approach may be applied to personalized medicine in two main ways: (1) to provide clinicians with useful prognostic information; and (2) as a decision-aid system to assist clinicians and patients in making decisions regarding anti-VEGF injections, including recommending or refusing injections, decisions regarding injection intervals, and the possible need for drug replacement. The algorithm will also help identify patients who do not respond well to anti-VEGF therapy, suggesting the need for close follow-up and timely adjustment of treatment strategies. More importantly, our prediction system is based on commonly used OCT images and a well-established algorithm. This does not require additional investment in new ophthalmic examination equipment, and artificial intelligence algorithms make the system available to a larger number of physicians.

Numerous studies have supported the potential broad applicability of our model. For instance, Sudeshna et al. extracted features from OCT images to distinguish between rebounders and non-rebounders after anti-VEGF treatment in DME/RVO-ME patients, achieving an AUC of 0.78 ± 0.08 [36]. Similarly, Mathias et al. used machine learning to predict the anti-VEGF treatment needs of patients with ME within 1 year, categorizing divided eyes into low, medium, and high-demand groups based on average treatment intervals, with the average AUC of the RVO/DME model being 0.76 for low demand and 0.78 for high demand [37]. These studies did not distinguish between RVO-ME and DME patients, and their results indicated the reliability of deep learning. This suggests that image features in various ME diseases may share similarities, making deep learning algorithms potentially applicable to different ME conditions, including diseases like DME. Our algorithm may thus find utility in a wider range of ME-related applications.

Our study had some limitations and improvements. First, although our results have some advantages, they are limited by their retrospective nature and small sample size, and only 140 patients were used for training and testing. A larger OCT dataset for training and testing is expected to yield better prediction results. Secondly, the response to treatment with the injectable anti-VEGF drugs in this trial, including aflibercept, ranibizumab, and bevacizumab, may vary. Third, the predictive ability is demonstrated by whether recurrence is present, no prediction of visual outcome is made, and the recovery of anatomical structures does not certainly indicate functional restoration. Fourth, our study did not identify the anatomical and pathological features that had a significant impact on the prediction results. A larger prospective observational trial using a standardized imaging protocol is required to confirm these findings.

5. Conclusions

Our deep learning model fused clinical features and successfully predicted recurrence within 3 months based on preprocessed OCT images in patients with RVO-ME. Predicting whether a relapse will occur after treatment based on our deep learning system may help better manage the treatment and follow-up of patients with RVO. Further development of the prediction system could be achieved by including larger cohorts and conducting long-term prospective cohort studies in the future. In summary, the artificial intelligence system established the feasibility of helping ophthalmologists identify patients most likely to experience recurrent RVO-ME.

Funding

This study was supported by the National Natural Science Foundation of China (grant number 81970823).

Data availability statement

Clinical and imaging data from The Affiliated Eye Hospital of Nanjing Medical University are not publicly available to protect patient privacy. The anonymized data and codes used in this study are available from the corresponding author, JY, upon request.

Ethics approval and consent to participate

This retrospective study was conducted from February 2018 to June 2022 and approved by the Ethics Committee of Nanjing Medical University Eye Hospital (No. 2023001). The requirement for informed consent was waived by the ethics review board due to the retrospective design of the study.

CRediT authorship contribution statement

Laihe Zhang: Writing – original draft, Data curation. Ying Huang: Writing – original draft, Data curation. Jiaqin Chen: Writing – original draft, Data curation. Xiangzhong Xu: Writing – review & editing. Fan Xu: Writing – review & editing, Software, Methodology, Data curation. Jin Yao: Writing – review & editing, Supervision, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article.