Classifying retinal images via vascular-optic disc cross-segmentation and attentive feature selection

Abstract

The prevalence of retinal disorders is rising at an alarming rate, posing a significant risk of irreversible blindness without timely intervention. Recent advancements in artificial intelligence (AI) have enabled the development of automated screening systems for retinal image analysis. However, building robust models presents challenges due to inconsistencies in image acquisition, varying resolutions, and imbalanced class distributions. To address these issues, this research introduces a novel abnormality-aware attentive feature selection approach for classifying retinal images into three categories: healthy (H), glaucoma (G), and diabetic retinopathy (DR). This method focuses on extracting the most discriminative features to enhance classification performance. Additionally, we propose a cross-segmentation framework that extracts and integrates comprehensive multiview information—specifically, optic disc and vascular structures—from the input retinal images. By leveraging these supplementary features, our approach effectively addresses low inter-class variability and enriches the information available for accurate classification. Extensive experiments conducted on diverse publicly available datasets—including FIVES, Drishti-GS1, SUSTech, HRF, and PAPILA—demonstrate the robustness of the proposed method. The model achieved a balanced accuracy of 83% (95% CI: 80%–86%), with sensitivities of 0.81 for the healthy class, 0.81 for glaucoma, and 0.87 for diabetic retinopathy. The corresponding specificity values were 0.84, 0.83, and 0.85; positive predictive values (PPV) were 0.83, 0.75, and 0.90; and negative predictive values (NPV) were 0.86, 0.91, and 0.96, respectively. These results indicate that the model is robust in distinguishing between the three classes, exhibiting balanced performance across all metrics.

Introduction

The retina, located in the eye, is a thin tissue layer composed of specialized cells called photoreceptors. These photoreceptors primarily detect light and transmit signals to the brain for interpretation¹. Retinal imaging techniques are employed to obtain detailed images of the retina’s structure. However, the retina is susceptible to damage from various factors such as ageing, diabetes, genetic disorders, inflammatory diseases, and infections. Retinal diseases develop gradually and typically do not exhibit noticeable symptoms. There is an increased emergence of patients suffering from retinal diseases^2,3. Among various retinal diseases, diabetic retinopathy and glaucoma are significant contributors to moderate to severe vision loss. DR affects the blood vessels in the retina and is particularly prevalent among individuals with diabetes. DR results in several abnormalities in retinal images, such as exudates, microaneurysms, cotton wool spots, and neovascularization distributed throughout the image⁴. The global prevalence of diabetes is substantial, with approximately 540 million people affected in 2023, and this number is projected to rise to 643 million and 783 million by 2030 and 2045, respectively⁵. Timely diagnosis is essential to prevent irreversible visual impairment. glaucoma, characterized by optic nerve damage⁶due to increased intraocular pressure, progresses slowly and gradually impairs peripheral vision, eventually leading to blindness if left untreated. As per the study⁷, it is estimated that glaucoma cases will rise to approximately 111 million by 2040. As both DR and glaucoma are progressive and irreversible conditions associated with vision loss, early detection and effective management are of utmost importance. The standard method to assess a retinal image is fundus photography. The systematic diagnosis of retinal diseases in resource-constrained regions faces significant challenges due to the high costs associated with retinal cameras, inadequate infrastructure, and a shortage of medical professionals⁸. Hence, there arises a need to develop automated solutions that can aid healthcare practitioners in the early screening process.

Given the prevalence of DR and glaucoma, this paper focuses on classifying a given fundus image into healthy, glaucoma or DR. Various studies have been conducted on the automated classification of retinal diseases^9–14. Arunkumar et al.¹⁵presented a methodology for classifying multiple retinal diseases by employing a deep belief network to extract relevant features from the data. Gour et. al¹¹. proposed an approach for multi-class and multi-label ocular disease detection. Nazir et al.¹⁶ proposed an approach that employed CenterNet for localizing and classifying retinal lesions, although it was computationally expensive due to the use of DenseNet-100 as the feature extractor. Zhu et al.¹⁷introduced an EfficientNet-based model for multi-retinal disease classification. Thanki¹⁸proposed an approach to detect glaucoma by evaluating a set of machine learning classifiers using deep features. In another study by Latif et. al¹⁹., the authors highlighted the significance of optic disc in classifying glaucoma. Lu et. al²⁰. present a transfer learning-based approach to classify retinal diseases. The authors used ShuffleNet-V2 and demonstrated that their proposed approach can be deployed on embedded devices²¹. proposed Fundus-DeepNet model that combines the feature representations from a set of fundus images for detecting multiple eye diseases. Several review papers provide a comprehensive overview of retinal image analysis, covering current trends in fundus image classification and segmentation^22–24. These reviews summarize the strengths and limitations of various approaches, noting, for instance, the suitability of UNet for medical imaging tasks and the performance improvements gained through techniques like Contrast Limited Adaptive Histogram Equalization for enhancing image quality. However, some limitations need research focus, such as the unavailability of large annotated datasets, the need for lightweight models suitable for deployment on edge devices, and the development of generalizable models to fill the gap due to varying acquisition devices. Furthermore, there is a need to develop approaches to help with early screening in resource-constrained areas. Addressing these issues is crucial to expanding the utility of retinal image analysis for broader and more equitable healthcare applications. To summarize, the main limitations of the existing approaches of retinal disease classification are listed as follows.

• These approaches often report results based on specific population groups, small datasets, or private test sets, with limited focus on unseen cross-dataset evaluations. Cross-dataset validation ensures robustness and generalizability across diverse populations and clinical settings.

• Many approaches rely solely on fundus images as input, which can lead to performance degradation due to poor image quality caused by low contrast or resolution variations. Addressing these issues with robust preprocessing and multiview inputs enhances model reliability.

• A thorough evaluation of the models across individual disease classes and diverse ethnic datasets is still lacking. Yet, it is essential to fully assess the effectiveness of the proposed methods across varied demographics.

To address these limitations, we propose a deep learning-based framework, combining multiple information modalities from the same retinal image, including the original fundus image, vascular structure, and the optic disc, to enhance the performance of retinal image classification. The core of the proposed method is abnormality-aware weighted feature selection and aggregation, which refines the final classification process by focusing on the most informative features, as shown in Fig. 1. To support this, we use a cross-segmentation framework called “VOxSeg” that extracts the required masks (vessels or optic disc) from the fundus images. Adding complementary information about the optic disc and vascular structure can help improve the classification^25,26. Feature selection plays an important role by filtering out irrelevant features, thus maintaining and enhancing classification performance^27–29. By selecting the most informative features, we reduce the number of features and mitigate the risk of performance degradation due to the curse of dimensionality, which can arise from using multiple inputs. This work uses datasets representing diverse ethnic groups to ensure robust retinal disease classification. The effectiveness of the proposed approach is demonstrated through extensive experimental evaluations, highlighting its performance across various metrics. The contributions of the proposed work are as follows:

1. We propose a novel multiview framework that integrates the original retinal image with optic disc and vascular structure information.

2. The proposed method uses a weighted, abnormality-aware feature-selective concatenation of the extracted features to enhance the feature representation.

3. A novel cross-segmentation framework called “VOxSeg” is proposed to predict the optic disc and vascular segmentation masks from the retinal images.

4. Extensive experiments were conducted across various performance measures, providing insights into the effectiveness of the approach on different datasets, potentially indicating its generalizability and robustness.

Fig. 1.

Visual concept drawing the contrast between the existing (i) method of image-based classification approach and (ii) the enhanced classification achieved through the proposed framework, highlighting the significance of boosted information and a feature selection technique sensitive to abnormalities.

Methods

Problem definition and proposed approach

This work focuses on retinal image analysis where a given image is classified as Healthy, glaucoma or DR. The proposed approach integrates additional information from the retinal vessels and optic disc, leading to significant improvement in the overall performance. Furthermore, we have evaluated the method across diverse ethnic groups to assess its robustness and generalizability.

Fig. 2(a) illustrates the proposed approach for the retinal image classification. The proposed methodology consists of two novel frameworks:

• A cross-segmentation framework called VOxSeg is introduced to predict the optic disc and vascular structure of retinal images. Optic Disc segmentation of retinal images is performed for the data samples that are utilized for training the vascular segmentation network and cross-wisely, vascular structure segmentation is performed for those retinal data samples that are utilized for training the optic disc segmentation network.

• Learnable Abnormality-aware Attentive Feature Selection approach is proposed which enables the model to extract and select the features that are influenced based on class information.

Fig. 2.

Showcasing the proposed architecture (a) for the weighted multiview retinal image classification and (b) Illustrating the preprocessing steps for enhancing the quality of the retinal image. In b(a), the retinal image is depicted, and b(b) displays the process of isolating colour channels. While b(c) illustrates the extraction of the green channel and its subsequent normalization, accompanied by the representation of their intensity distributions. b(d) Showcases the resulting clahe image and its corresponding intensity distribution.

The datasets used in this work come from different locations, and they exhibit variations in resolution, quality, number of samples per class, and the devices used for acquisition, as given in Table 1. Hence, it is necessary to preprocess all the images to ensure proper retinal image analysis. To handle the class imbalance problem, we applied augmentations such as linear contrast adjustment, left/right rotation, and sharpness enhancement to the training data belonging to the diseased classes. However, since the healthy class contains an adequate number of samples, no augmentation was applied to this class. Excessive augmentation can result in poor generalizability, as it repeats the original samples with minor variations. To align the training samples of the diseased class with the healthy class during training, we used an under-sampling strategy to ensure that the number of healthy class samples is proportional to the other classes.

Table 1.

(a) Characteristics of the datasets used in the proposed approach and (b) Performance metrics used to assess the proposed approach. True Positive (TP): Number of samples correctly classified as positive, False Positive (FP): Number of samples incorrectly classified as positive, True Negative (TN): Number of samples correctly classified as negative, False Negative (FN): Number of samples incorrectly classified as negative, Balanced Accuracy (BACC): A measure of average accuracy across all classes, giving equal importance to each class, Positive Predictive Value (PPV) represents the proportion of true positive predictions (correctly predicted positive instances) among all instances predicted as positive, Negative Predictive Value(NPV) represents the proportion of true negative predictions (correctly predicted negative instances) among all instances predicted as negative, Kappa Statistic (K) quantifies the agreement between classifier and true labels, adjusting for chance agreement. Mean Intersection over Union (mIoU) measures the overlap between predicted mask (A) and ground truth mask (B) and Mean Dice Coefficient (mDC) measures the similarity between predicted mask (A) and Ground truth mask (B).

Dataset	Origin	H	G	DR	Vascular Mask	Optic Disc Mask
PAPILA40	Spain	333	155	0	No	Yes
HRF41	-	15	15	15	Yes	No
SUSTech42	China	631	0	588	No	Yes
FIVES43	China	200	200	200	Yes	No
Drishti-GS144,45	India	50	51	0	No	Yes

Metric	Formula
Precision	TP / (TP + FP)
Recall	TP / (TP + FN)
F1 Score	2 x (Precision x Recall) / (Precision + Recall)
Sensitivity	TP / (TP + FN)
Specificity	TN / (TN + FN)
Classwise Accuracy (CA)	TP / (TP + FP)
Balanced Accuracy (BACC)	(TP1 + TP2 +... + TPn) / N
Positive Predictive Value (PPV)	TP / (TP + FP)
Negative Predictive Value (NPV)	TN / (TN + FN)
Kappa Statistic (K)
Mean Intersection over Union (mIoU)
Mean Dice Coefficient (mDC)

Since the acquired retinal images are prone to low contrast and uneven illumination, it is necessary to preprocess them to obtain the desired image for a specific task³⁰. Figure 2(b) illustrates the preprocessing stages that enhance the quality of the input retinal image. For the segmentation task, we extracted the green channel of the input image, as it is believed to contain the most detailed information. Subsequently, we applied the Contrast Limited Adaptive Histogram Equalization (CLAHE)^31–33 technique, which enhances the overall contrast of the image. To address variations in resolution, all images were normalized and resized to a uniform size of 224x224 pixels. This normalization not only ensures consistent resolution but also helps reduce computational complexity.

VOxSeg: Vascular-optic disc cross-segmentation

The vascular structure in the retinal image is important for identifying different retinal conditions such as DR, glaucoma, and age-related macular degeneration³⁴. Similarly, the information within the optic disc is vital for diagnosing glaucoma^35,36. Many studies in the literature for glaucoma diagnosis use segmented optic discs of the retinal image for classification^35,37. However, these approaches are limited to a single disease. Therefore, it is necessary to analyze the original whole retinal image to develop a more adaptable method for classifying various retinal diseases. This paper proposes a novel approach that combines complementary infromation from the optic disc, vascular information, and the original fundus image to enhance diagnostic performance. Therefore, by incorporating additional information, the diagnostic performance of the model is improved. However, finding this type of information is challenging in existing datasets, and manual annotation is often very time-consuming.

To address this limitation, we propose a novel technique called VOxSeg, a cross-segmentation approach designed to extract the required masks (optic disc or vascular structure) from the datasets used in this work. Since not all datasets provide ground truth segmentation masks for vessels or optic discs as outlined in Table 1, we use the available data in each dataset for their respective tasks. We train the Vascular-UNet model on the HRF and FIVES datasets (set 1) for vessel segmentation and the OD-UNet model for optic disc segmentation using the SUSTech, Drishti-GS1, and PAPILA datasets (set 2). The segmentation loss between ground truth mask (A) and predicted mask (B) computed using Dice loss is presented as equation (1).

1

Segmentation Mask Generation: After training, the segmentation block, including OD-UNet and Vascular-UNet models, is integrated into the proposed pipeline with frozen weights to generate segmentation masks for all input images automatically. The proposed VOxSeg approach effectively extracts the optic disc for set 1 using OD-UNet and vascular structure for set 2 using Vascular-UNet. This cross-segmentation approach enables the prediction of all required masks for both sets, significantly reducing the need for manual annotation. Using this method, we ensure that all necessary features, such as optic disc and the vascular structure, are consistently available for both sets, thus improving the robustness and adaptability of the diagnostic pipeline across various datasets.

The internal structures within the optic disc carry valuable diagnostic information. Hence, it is essential to isolate and analyze these features. We multiply the original fundus image with the binary mask obtained from the OD-UNet model to extract the internal structures of the optic disc image (OdI). This postprocessing enhances the visibility of the critical features within the optic disc, allowing for a more focused analysis. Unlike the optic disc mask, the vascular mask is employed directly within the proposed pipeline, as the morphology and integrity of the retinal blood vessels can reflect the severity of the eye disease. Subsequently, all the images, including the original retinal image, vascular mask, and the extracted region of the optic disc, are fed into the feature selection module for further processing. This comprehensive integration of multiview information from both the optic disc and vascular structures enhances the diagnostic performance of the model by providing a richer set of features for disease classification.

Learnable abnormality-aware attentive feature selection

Various abnormalities necessitate focusing on distinct regions of interest (ROI) within a fundus image, with the selection of these regions depending on the condition being analyzed. For example, glaucoma requires a greater focus on the optic disc region, whereas Diabetic Retinopathy requires a stronger emphasis on the vascular region in the image. To incorporate this concept into the proposed pipeline, we propose an optimal classwise feature selection for the features extracted from multiple inputs - the fundus image, the extracted optic disc, and the vascular mask. This optimal feature selection identifies the relevant feature values from the individual embeddings and minimizes the effect of redundant features. Also, we propose using learnable parameters to guide the feature selection process, enabling the attentive fusion of features corresponding to the output classes. This novel approach helps to identify optimally weighted features, thus enhancing the efficiency of retinal disease classification.

We used the EfficientNet architecture for feature extraction as it requires fewer computational resources³⁸. Let , where 1, 2, 3, denote the three feature extractors: RetinalImg-Net, OpticDisc-Net and Vascular-Net, respectively. The input X refers to a retinal image (I), an extracted optic disc (OD), and a vascular mask (V). RetinalImg-Net generates feature embeddings () from the original retinal image (I), while OpticDisc-Net takes OdI as input to generate feature embeddings (). Meanwhile, Vascular-Net generates feature embeddings () using the predicted binary vascular mask (V). The embeddings of each network are defined as:

2

The embeddings encompass diverse feature values that capture their contributions in classifying the output classes. For this purpose, the three networks are utilized to extract meaningful features from all three images: retinal image, predicted optic disc, and vessel segmentation masks. We performed end-to-end training of all three networks to predict the disease or abnormality by classifying it into three classes: healthy (H), glaucoma (G), and diabetic retinopathy (DR). These three networks are trained together in an end-to-end fashion in order to obtain the relevant features from all three images, which will be useful together to obtain a highly informative representation that helps to differentiate between different classes, leading to correctly classifying the diseases.

A feature selection technique addresses that certain features demand more attention than others to focus on specific image regions. Equation (3) formulates the utilization of the L1-feature selection technique in this work. The L1-feature selection method encourages sparsity by enforcing zero coefficients for weak features while retaining non-zero coefficients to introduce penalties in the loss function. This leads to sparse solutions that effectively highlight the most relevant features. Incorporating sparse learning into feature selection involves identifying a transformed matrix, denoted as where d refers to the embedding dimension and k is the number of classes. This matrix evaluates the relationship between labels and features, assigning weights to determine the most informative features. Through this process, the feature selection technique discovers the transformed matrix w and selects the relevant feature values based on their assigned weights.

3

where refers to the weights between each feature embedding and the classification layer. The feature selection is applied to feature embeddings extracted to analyze the importance of each feature in classifying the image as healthy or diseased (Glaucoma or Diabetic Retinopathy). The contribution of each feature embedding varies in its weight corresponding to the output classes, with strength parameters assigned to each embedding based on their relevance to the particular abnormality. This strength is achieved through the learnable parameters , and a hyperparameter . The and automatically determine the contribution of the optic disc image and vascular information in classifying the abnormalities more precisely. Meanwhile, for the healthy class, all inputs require equal attention; thus, gamma is chosen as the hyperparameter, ensuring balanced weights across all feature embeddings.

The values of strength parameters help to define the impact of each feature embedding in achieving accurate classification for each class. Algorithm (1) elucidates the proposed algorithmic formulation for Learnable Abnormality aware Attentive Feature Selection.

Algorithm 1.

Learnable Abnormality aware Attentive Feature Selection

Proposed loss function

The proposed loss function is composed of three components: cross-entropy loss (), Learnable Abnormality-Aware Weight Assignment Loss (), and a regularization term (). Each term contributes to different aspects of the learning process, as explained below.

1. Cross-Entropy Loss (): The cross-entropy loss () as shown in equation (4) is used to classify between healthy, glaucoma and DR classes.

   4

   where, k represents the number of classes, refers to the true label for the class and is the softmax probability for the class.
2. Feature Selection Regularization Loss (): This term, calculated as shown in equation (5), supports efficient feature selection. It selectively retains relevant features from each embedding while suppressing the less relevant ones.

   5
3. Learnable Abnormality-Aware Weight Assignment Loss (): This loss term, shown in equation (6), assigns weights to the embeddings by learning the values of learnable parameters and . Since different retinal diseases require focus on specific regions, these parameters adapt to prioritize relevant regions. Since glaucoma needs more attention towards the optic disc region than the vascular region, and DR requires more emphasis on the vascular part, this helped in setting the value of and .

   6

   Here, ’s are the adjustable factors that bring the overall loss function () into a similar range.
   The final combined loss used for training the proposed Learnable Abnormality-aware Attentive Feature Selection approach is defined as equation 7.

   7

It is important to note that some cases may involve patients suffering from multiple diseases, such as glaucoma and DR. A model using a softmax function at the classification layer would only identify one disease in these cases, as softmax is designed for single-label classification. We have used the sigmoid function at the classification layer to accommodate such cases and trained the model with a binary cross-entropy loss for each classification node. This approach enables the model to identify and output both diseases if present. The summarized results for the model trained with binary cross-entropy loss and the application of the sigmoid function at the classification layer are reported in Table 4.

Table 4.

Performance comparison between healthy and each diseased class by adopting a sigmoid activation function at the classification layer. (CA: classwise accuracy (%), BACC: Balanced Accuracy (%)).

Input		Class	Precision	Recall	F1 Score	CA	Specificity	Sensitivity	BACC
Simple Images	H vs G	H	0.92	0.83	0.87	82.63	0.82	0.87	84.99
		G	0.74	0.87	0.80	87.35
	H vs DR	H	0.96	0.94	0.95	93.56	0.93	0.94	93.85
		DR	0.90	0.94	0.92	94.14
Concatenated Images	H vs G	H	0.95	0.68	0.79	67.52	0.97	0.93	80.31
		G	0.62	0.93	0.74	93.10
	H vs DR	H	0.94	0.83	0.88	82.95	0.82	0.91	87.21
		DR	0.76	0.91	0.83	91.48
With OD	H vs G	H	0.93	0.82	0.87	81.67	0.81	0.89	85.66
		G	0.73	0.90	0.81	89.65
	H vs DR	H	0.96	0.91	0.94	90.99	0.90	0.94	92.56
		DR	0.86	0.94	0.90	94.14
With vessels	H vs G	H	0.87	0.80	0.83	80.06	0.80	0.78	77.89
		G	0.69	0.79	0.73	75.73
	H vs DR	H	0.93	0.93	0.93	92.60	0.92	0.88	90.71
		DR	0.88	0.89	0.88	88.82
Without feature selection	H vs G	H	0.89	0.81	0.85	81.35	0.81	0.82	82.05
		G	0.71	0.83	0.77	82.75
	H vs DR	H	0.94	0.92	0.93	91.63	0.91	0.89	90.76
		DR	0.87	0.90	0.88	89.89
Proposed* (: 1.0, : 1.0, : 1.0)	H vs G	H	0.95	0.71	0.81	71.38	0.71	0.93	82.24
		G	0.65	0.93	0.76	93.10
	H vs DR	H	0.97	0.88	0.92	88.42	0.88	0.95	92.02
		DR	0.83	0.95	0.89	95.21

Implementation details

We used two UNet³⁹models, Vascular-UNet and OD-UNet, to perform segmentation tasks. Vascular-UNet focused on segmenting vessels, while OD-UNet focused on optic disc segmentation. The segmentation models were trained for 70 epochs using the Adam optimizer, with the learning rate at 0.001. EfficientNet_B4³⁸ is used as the feature extractor across all inputs for the classification task. A fully connected layer was added to the model to process and select the combined features for the final prediction. The proposed model was trained for 50 epochs using the Adam optimizer with a weight decay of 1e-3. The learning rate was set to 0.0001. Both tasks used a batch size of 32, and images were resized to 224x224x3. The experiments were conducted on an Nvidia DGX station consisting of four V100 GPU cards, each with 32GB of memory and 256GB of RAM. The proposed algorithm was implemented in PyTorch.

Dataset details

To evaluate the proposed approach, we utilized publicly available datasets. Table 1(a) gives the dataset details, and the characteristics of each dataset are summarized below.

PAPILA⁴⁰ The dataset comprises 488 fundus images representing both eyes of 244 patients. The images are captured using a TOPCON TRC-NW400 retinograph, providing a resolution of 2576 x 1934 pixels, with a 45-degree opening angle, and taken in a frontal and centered view. An ophthalmologist annotates each image for the optic disc and optic cup. Additionally, the dataset includes classification labels for each retinal image, with 333 images classified as healthy, 87 images identified as confirmed glaucoma cases, and 68 images marked as suspected cases of glaucoma.

Drishti-GS1^44,45 It consists of 101 retinal images pertaining to healthy and glaucomatous classes. The dataset is divided into train and test splits of 50 and 51 images. Each image is captured with a 30-degree field of view and centered on OD. All the images are provided with annotations of the optic disc and optic cup.

SUSTech⁴² The dataset consists of 1219 fundus images categorized into 631 images of the healthy class and 588 images of the DR class. Each image has a resolution of 2880 x 2136 pixels. The images are captured using a Topcon TRC-50DX fundus camera, covering OD and macula regions.

HRF⁴¹ This database is composed of 45 fundus images with a high resolution of 3504 2336 pixels and is evenly distributed across three classes: 15 images of healthy patients, 15 images of patients with diabetic retinopathy and 15 images of glaucomatous patients. A binary vessel segmentation ground truth accompanies each image in the database.

FIVES⁴³ The dataset consists of 800 fundus images with a resolution of 2048 x 2048 pixels, obtained from 573 patients. The images are captured using tabletop TRC-NW8 fundus cameras, providing a field of view of and centered at the macula region. The dataset includes 400 images of DR and 400 images of glaucoma. Furthermore, a team of annotators has provided pixel-wise annotation for the retinal vessels for each image.

The training images from all datasets were combined, with any dataset-specific test data kept separate to preserve integrity. A train-test split of 80:20 was performed. Furthermore, 10% of the training set was allocated as the validation set.

Experimental results

This section presents a detailed analysis of the results of the proposed approach. We evaluated the model using a set of metrics for segmentation tasks and classification as given in Table 1(b). We report the Mean Intersection over Union (mIoU) and Mean Dice Coefficient (mDC) for segmentation. We use precision, recall, F1 Score, sensitivity, specificity, classwise accuracy and balanced accuracy for classification, which are useful for medical imaging tasks. We have also reported the Positive Predictive Value (PPV), Negative Predictive Value (NPV) and Kappa Statistic (K), which highlights the reliability in model predictions and consistency of predictions across different classes.

VOxSeg: Vascular-optic disc cross-segmentation

Several retinal images suffer from poor contrast, making it challenging to accurately identify the optic disc and vessels. To address this issue, CLAHE is applied to the green channel of these images, as this is known to contain maximum information about retinal structures. This enhancement step improves the contrast and, hence, the quality of the segmentation outputs. The mIoU and mDC values for the OD-UNet are and , respectively, and for the Vascular-UNet model, the values are and , respectively. These scores indicate the effectiveness of the UNet architecture in accurately segmenting the optic disc and vessels. Fig. 3 shows the visuals of segmentation masks of pre-processed retinal images obtained through Vascular-UNet and OD-UNet models and their ground-truth masks. These images illustrate that OD-UNet and Vascular-UNet effectively learn segmentation patterns, producing masks that closely resemble the ground truth masks. The proposed VOxSeg approach depicts good segmentation performance even for low-contrast input retinal images. The slight irregularities in the boundaries of the predicted optic disc are due to the shading or reflection in the input image. However, it completely extracts the optic disc from the input image. The tiny blood vessels are not distinctly delineated due to the reduction in input image dimensions to . This resizing is necessary to manage computational demands and facilitate real-world deployment of the model. Furthermore, as indicated in⁴⁶, using larger input sizes can impose significant memory constraints, increase computational costs, and result in higher processing times. The authors in⁴⁷ further demonstrated that increasing image resolution yields only marginal gains in performance for tasks like diabetic retinopathy severity prediction.

Fig. 3.

Segmentation results on datasets (a) input image, (b) preprocessed image, (c-i) and (c-ii) represents ground truth optic disc and ground truth vessels mask, respectively, (d-i) and (d-ii) represents predicted optic disc and predicted vessels, respectively and (e-i) and (e-ii) represents generated vessels and optic disc using Vascular-UNet and OD-UNet, respectively.

Learnable abnormality-aware attentive feature selection

We have conducted a comprehensive analysis using various existing architectures and cross-dataset evaluations. This comparison evaluates the proposed approach against existing models, with results based on our implementation of their architectures. Dataset-specific results are also reported to demonstrate the generalizability and robustness across diverse population groups. For visual interpretation of the model predictions, we present explainability maps generated using the GradCAM technique. Statistical validation with various metrics is also used to assess the model performance.

The proposed abnormality-aware feature selection technique effectively identifies the most relevant features by shrinking the coefficients of less important features to zero, thereby eliminating less relevant features. After applying the feature selection, the number of features are reduced from 768 to 657 for predicting the healthy class, 642 for glaucoma prediction, and 654 for diabetic retinopathy. The proposed approach outperforms other architectures in terms of balanced accuracy, as shown in Table 2. The effectiveness is further evaluated in terms of precision, recall, F1 Score, and classwise accuracy metrics. The proposed method achieves a balanced accuracy of when classifying healthy, glaucoma, and DR cases, outperforming all other approaches. Additionally, the proposed algorithm achieves specificities of 0.85, 0.89, and 0.96, and sensitivities of 0.81, 0.81, and 0.87, which are more evenly distributed across the classes, indicating balanced learning across all classes. These sensitivity and specificity values result in an F1 Score of 0.82, 0.78, and 0.89, further affirming the proposed method’s effectiveness for these classes. Moreover, other performance metrics such as precision, recall, and classwise accuracy also highlight the efficiency of the proposed approach.

Table 2.

Performance comparison of existing architectures and varying strategies with the proposed approach on the combined dataset. (CA: classwise accuracy (%), BACC: Balanced Accuracy (%)).

Input	Class	Precision	Recall	F1 Score	Sensitivity	Specificity	CA	BACC
Existing Architectures
E417	H	0.85	0.78	0.81	0.77	0.87	77.81	80.89
	G	0.68	0.80	0.74	0.79	0.86	79.88
	DR	0.86	0.85	0.86	0.85	0.93	85.10
SNet20	H	0.85	0.68	0.75	0.67	0.89	67.84	80.34
	G	0.60	0.90	0.72	0.89	0.77	89.65
	DR	0.96	0.84	0.89	0.83	0.98	83.51
EyeNET48	H	0.67	0.45	0.54	0.45	0.75	45.33	53.33
	G	0.37	0.54	0.44	0.54	0.61	54.02
	DR	0.55	0.61	0.57	0.60	0.71	61.63
Concatenated Images	H	0.87	0.70	0.78	0.70	0.89	70.40	78.56
	G	0.58	0.84	0.68	0.84	0.77	83.90
	DR	0.92	0.81	0.86	0.81	0.96	81.40
Ablation Work
With OD	H	0.87	0.75	0.81	0.75	0.89	75.24	82.25
	G	0.71	0.81	0.76	0.81	0.87	81.03
	DR	0.84	0.91	0.87	0.90	0.91	90.95
With vessels	H	0.83	0.81	0.82	0.81	0.85	81.35	82.67
	G	0.75	0.79	0.77	0.78	0.90	78.73
	DR	0.88	0.88	0.88	0.87	0.94	87.76
Without feature selection	H	0.81	0.84	0.83	0.84	0.82	84.24	81.02
	G	0.75	0.78	0.77	0.78	0.90	78.1
	DR	0.90	0.81	0.85	0.80	0.96	80.85
Proposed Approach
Proposed* (, )	H	0.83	0.81	0.82	0.81	0.85	81.00	83.0
	G	0.75	0.81	0.78	0.81	0.89	81.00
	DR	0.90	0.87	0.89	0.87	0.96	87.00
Proposed* (: 0.87, : 0.782, : 1.0)	H	0.91	0.78	0.84	0.78	0.93	78.14	85.23
	G	0.69	0.91	0.79	0.91	0.85	91.38
	DR	0.92	0.86	0.89	0.86	0.96	86.17

To highlight the importance of integrating complementary information of vessels and optic disc, we evaluated the proposed approach by considering each component individually. The results are reported in Table 2. It is shown that incorporating optic disc information enhances sensitivities to 0.75, 0.81, and 0.90, while integrating vascular information improves specificities to 0.90, 0.94, and 0.82. Combining the advantages of OD and Vascular information, the proposed approach achieves an overall improved performance across all evaluation metrics compared to existing methods. We also provided results without feature selection, which yielded comparatively inferior performance compared to the proposed approach. Also, it has effectively classified the DR class and is often confused between healthy and glaucoma classes. This is primarily attributed to low inter-class and high intra-class variability between healthy and mild glaucoma cases. The proposed approach alleviates these issues by employing feature selection to choose more discriminative features and eliminate redundant ones. This leads to more balanced learning across classes and enhances the overall classification performance.

To showcase the robustness and generalizability of the proposed approach across diverse populations and imaging variations (Table 1), we computed dataset-wise performance metrics presented in Table 3. The results clearly indicate that the proposed approach shows good classification performance even when applied to datasets that do not contain all the classes used during training. We evaluated the proposed approach on DR test samples from the MESSIDOR2⁴⁹and RFMiD⁵⁰ datasets and got a sensitivity of 90% for MESSIDOR2 and 94% for RFMiD. These results showcase the model’s high sensitivity across unseen data, highlighting its robustness in varied clinical settings.

Table 3.

Dataset-wise performance metrics of the proposed approach (:1.0, :1.0 ,:1.0).

Dataset	Class	Precision	Recall	F1 Score	Sensitivity	Specificity	Accuracy(%)
PAPILA40	H	0.88	0.69	0.78	0.69	0.73	69.1
	G	0.46	0.73	0.59	0.73	0.69	73.3
	DR	0	-	-	-	1.0	-
HRF41	H	0.67	0.67	0.67	0.67	0.9	66.7
	G	0.88	1.0	0.94	1.0	0.8	100
	DR	0.67	0.67	0.67	0.67	1.0	66.7
SUSTech42	H	0.91	0.94	0.92	0.94	0.90	93.5
	G	1.0	1.0	1.0	1.0	1.0	100
	DR	0.89	0.89	0.89	0.89	1.0	89.4
FIVES43	H	0.71	0.8	0.75	0.8	0.82	80.0
	G	0.82	0.73	0.78	0.73	0.91	73.4
	DR	0.71	0.85	0.78	0.85	0.93	85.2
Drishti-GS144,45	H	0.73	0.53	0.63	0.53	0.93	53.3
	G	0.85	0.93	0.89	0.93	0.53	93.0
	DR	0	-	-	-	1.0	-

To showcase the applicability of the proposed approach in handling comorbid cases of DR and glaucoma, we performed additional experiments using binary cross-entropy loss at each classification node. This configuration enables the model to address cases with multiple diseases by applying a sigmoid function at the classification layer. The model can provide accurate multi-disease predictions by applying a threshold to the sigmoid output at each classification node. Table 4 presents the results for this setting, comparing the model’s performance in classifying healthy vs. glaucoma and healthy vs. DR cases. The proposed model achieved sensitivity values of 0.93 in the healthy vs. glaucoma classification and approximately 0.95 in the healthy vs. DR classification, underscoring its capability to accurately detect glaucoma and DR cases. Also, the model achieved specificities of 0.71 for the healthy vs. glaucoma case and 0.88 for the healthy vs. DR case, reflecting its effectiveness in correctly identifying individuals without these diseases. These results highlight the diagnostic strength of the proposed sigmoid-based approach, particularly for applications involving multi-disease detection.

After conducting extensive experiments with various hyper-parameter values, the values for and in the loss function are set to 1.0 since and need to be learned in range [0.5, 1]. The strengthening parameters and are initialized with 0.70 and 0.60 because assigns more strength to the embeddings as compared to , necessitating that be greater than . We also set the value for to 1. If the aim is balanced learning across classes, the learnt value of and will be set to 1. This proposed model having ( = ) ensures that equal importance is given to all extracted feature embeddings. However, if the aim is to predict the diseases more precisely, then the learned values of and are 0.87 and 0.782, respectively. In this model having (:0.87, :0.782 ,:1.0), the focus is shifted to the identification of diseased patients while slightly compromising on detecting healthy patients correctly. This model might be desirable in a medical setting, where the accurate identification of diseased patients is of paramount importance. Therefore, it will prevent false negatives and identify potential cases for further examination.

To provide visual explanations for the model’s decisions, GradCAM (Gradient-weighted Class Activation Mapping)⁵¹ is used. GradCAM highlights the importance of different regions within an image that contribute most significantly to a model’s predictions. GradCAM generates class-specific heatmaps by using the gradients of the class scores with respect to the feature maps obtained from the final convolutional layer. The original image is overlaid with a colour gradient from red to blue. In this context, red represents regions with increased emphasis, while blue represents regions with decreased focus. Fig. 4 presents examples of correctly classified instances for the three classes. The model focuses on the most discriminating area from the entire image for the healthy class and concentrates on or near the optic disc for the glaucoma class, which is more connected to the physical screening of the diagnosis of the glaucoma class. For the DR class, the model focuses on abnormalities such as microaneurysms, haemorrhages, and soft and hard exudates, aligning with known pathological features that are indicative of the disease. Adding visual explanations supports the applicability of the proposed approach in medical settings, as it provides meaningful insights into the model’s decision-making process. Fig. 5 displays examples of misclassified samples. As it is visually evident, the misclassifications primarily arise from factors caused by the imaging device, which incorporates false artefacts resembling any abnormality, insufficient contrast making it difficult to diagnose the true class, shading or may be due to class variations, especially noticeable in cases of healthy images or those depicting mild disease symptoms.

Fig. 4.

Demonstrating visual classification results alongside corresponding GradCAM visualizations of the correctly classified samples highlighting regions of diagnostic importance akin to human experts.

Fig. 5.

Exhibiting the incorrectly predicted samples of retinal images using the proposed framework.

To comprehensively assess the validation of the proposed approach, we computed several key metrics, such as C-statistic (AUC), Positive Predictive Value (PPV), Negative Predictive Value (NPV), and Cohen’s Kappa (K). The PPV values for healthy, glaucoma, and DR were 0.83, 0.75 and 0.90, respectively. The NPV values also highlight robust performance, with 0.86 for healthy, 0.91 for glaucoma, and 0.96 for DR. The Cohen’s Kappa value was 0.73, indicating substantial agreement between the model’s predictions and the ground truth labels. The overall accuracy and AUC of the model was 83% and 0.94, respectively, with a 95% confidence interval of [0.80–0.86], indicating excellent discriminative ability. These metrics highlight the reliability of the proposed model in classifying retinal diseases and demonstrate its applicability in clinical settings.

Performance with randomized cross-validation: We performed three iterations of a randomized (Monte Carlo) cross-validation approach, in which the dataset was randomly sampled to obtain different training and validation splits. This method provides a more robust evaluation of the proposed approach by assessing performance across multiple data partitions. We retained the same predefined test set (as described in the “Dataset Details” section) to ensure consistency when comparing the models. For the model with , , and , the average balanced accuracy on the test set was 82.44% with a standard deviation of . In contrast, the model with learnable parameters and achieved an average balanced accuracy of 84.03%, with a standard deviation of . These results demonstrate the reliability and generalizability of the proposed method.

Discussion

The prevalence of retinal diseases has shown a concerning increase in recent years, raising alarm within the global healthcare community due to the potential progression to irreversible blindness if left untreated. Artificial intelligence (AI) has emerged as a powerful tool with the potential to revolutionize the screening and diagnosis of retinal diseases. Automated methods for analyzing retinal images have created new opportunities for early detection and intervention. The application of AI in retinal image analysis necessitates the development of advanced algorithms capable of distinguishing between pathological conditions based on retinal scans. These models are trained on extensive datasets of retinal images along with their corresponding clinical diagnoses.The models are trained to learn patterns, features, and anomalies in retinal images that can be indicative of a disease. While the integration of AI into retinal disease screening holds great promise, it is important to acknowledge the challenges and limitations. One of the primary limitations is the variability in image acquisition. The images can be captured using a range of devices exhibiting variations in resolution, lighting conditions etc.These variations can affect the consistency and reliability of AI models. Another significant challenge is the issue of class imbalance where certain retinal diseases may be less prevalent than others. This class imbalance can cause bias towards prioritizing accuracy in identifying the classes with more significant samples over others. Researchers are actively working on developing AI models to address these challenges. However, existing approaches have limitations in terms of their generalizability and reliance on single inputs.

To address these limitations, the proposed approach presented two contributions- First, VOxSeg (Vascular-Optic disc Cross-segmentation), a cross-segmentation framework for predicting optic disc and the vascular structure from the input image is proposed. This framework enables us to extract a multiview information from the single input image. Second, we propose a weighted abnormality-aware feature selection-based multiview approach for retinal image classification which takes into account the importance of different features and their contributions to the final classification. By integrating the optic disc and vascular structure information alongside the original retinal image, the performance is significantly improved. The proposed approach undergoes thorough evaluation, assessing the importance and impact of each component employed in the proposed pipeline. Moreover, visual explanations are provided to interpret the model’s decision-making process.

Given the unique dataset protocol used in this paper, a comparison was conducted between the proposed method and existing models developed for other multiple retinal diseases. ENet¹⁷employs transfer learning with EfficientNet-B4 as the backbone for retinal disease classification. However, their approach tends to favour one class over others, resulting in reduced generalizability when faced with increased classes or small inter-class variability. SNet²⁰, on the other hand, propose using ShuffleNet-V2 as the backbone for feature extraction. While their approach performs well in classifying diseased classes, it struggles in detecting healthy classes. EyeNET⁴⁸ proposes a customized network for multiple retinal image analysis but fails to handle small inter-class variability between glaucoma and healthy classes. In contrast, the proposed approach demonstrates good generalization performance in terms of balanced accuracy, sensitivity, and specificity across all classes. The performance of the proposed approach is more evenly distributed, indicating balanced learning among all classes. Evaluating with individual test sets yields good classification results, even with varying acquisition locations, ensuring generalizability. The DR class exhibits high specificity across all approaches, likely due to the visual distinctiveness of DR samples, as mentioned in the original paper’s dataset details. However, glaucoma and healthy classes exhibit misclassifications, resulting in low sensitivity. This misclassification can be attributed to the ocular similarities between mild or low glaucoma cases and the healthy class, making it difficult for even trained professionals to distinguish between the two in real-world situations. More advanced segmentation and feature extraction approaches will be required to better handle minute vessels and small inter-class variability.

In conclusion, the proposed approach demonstrates an effective method for classifying retinal diseases. The novel cross-segmentation framework, VOxSeg, enables the extraction of multiview information from the input retinal image, resulting in a rich feature set for the classification pipeline. Also, the weighted abnormality-aware feature selection process allows removing redundant features while focusing on those critical for each disease class. This helps address class variation and enhances overall classification performance. The experimental results further validate the efficacy of the proposed approach. Furthermore, the diverse datasets used in this work contribute to the generalizability and robustness of the methodology, ensuring its applicability across various populations and conditions.

Author contributions

All the authors contributed to the problem formulation. M.K. and C.C. conceived the methodology and conducted the experiments. All the authors analyzed the results and contributed towards drafting, reviewing, and editing the manuscript.

Data availability

The datasets analyzed and used during the current study are publicly available. The datasets are available at the given links PAPILLA- (Link), FIVES- (Link), SUSTECH-SYSU- (Link), HRF - (Link), and Drishti-GS1 is available from the corresponding author on reasonable request using the dataset portal.

Declarations

Competing interests

The authors declare no competing interests.