Domain Adaptation-Based Deep Learning Model for Forecasting and Diagnosis of Glaucoma Disease

Abstract

The main factor causing irreversible blindness is glaucoma. Early detection greatly reduces the risk of further vision loss. To address this problem, we developed a domain adaptation-based deep learning model called Glaucoma Domain Adaptation (GDA) based on 66,742 fundus photographs collected from 3272 eyes of 1636 subjects. GDA learns domain-invariant and domain-specific representations to extract both general and specific features. We also developed a progressive weighting mechanism to accurately transfer the source domain knowledge while mitigating the transfer of negative knowledge from the source to the target domain. We employed low-rank coding for aligning the source and target distributions. We trained GDA based on three different scenarios including eyes annotated as glaucoma due to 1) optic disc abnormalities regardless of visual field abnormalities, 2) optic disc or visual field abnormalities except ones that are glaucoma due to both optic disc and visual field abnormalities at the same time, and 3) visual field abnormalities regardless of optic disc abnormalities We then evaluate the generalizability of GDA based on two independent datasets. The AUCs of GDA in forecasting glaucoma based on the first, second, and third scenarios were 0.90, 0.88, and 0.80 and the Accuracies were 0.82, 0.78, and 0.72, respectively. The AUCs of GDA in diagnosing glaucoma based on the first, second, and third scenarios were 0.98, 0.96, and 0.93 and the Accuracies were 0.93, 0.91, and 0.88, respectively. The proposed GDA model achieved high performance and generalizability for forecasting and diagnosis of glaucoma disease from fundus photographs. GDA may augment glaucoma research and clinical practice in identifying patients with glaucoma and forecasting those who may develop glaucoma thus preventing future vision loss.

1. Introduction

Glaucoma disease is a leading cause of irreversible blindness worldwide [3]. Early detection is an effective approach to slow glaucoma and prevent subsequent vision loss [4]. However, detecting glaucoma requires extensive clinical examination and assessment of imaging and visual field tests. Fundus photography is a convenient and non-expensive technique for recording the status of the optic nerve head structure [5–8]. Although, detecting glaucoma from fundus images requires there is a considerable disagreement in diagnosis even among experienced glaucoma experts [4, 9]. Therefore, developing highly accurate, dependable, and low-cost automated models to detect glaucoma from fundus images is critical in ophthalmology.

Several deep convolutional neural network (CNN) models have been introduced to automatically identify various ocular diseases such as diabetic retinopathy, age-related macular degeneration, and glaucoma [10–13]. Deep CNN models typically require massive, annotated datasets for training. Additionally, using training and testing subsets with equivalent distributions is a pre-requisite for these models; otherwise the accuracy may be compromised [14]. Since data annotation is a challenging and labor-intensive task, particularly in the healthcare field, domain adaptation (DA) techniques may be used to deal with these situations in which limited numbers of annotated data exist. Also, in real-world applications, the training and testing samples may often be extracted from dissimilar sources (e.g., fundus photographs collected from subjects of different races in different countries) leading to different distributions. DA models can address such problems by transferring knowledge from training (source domain) to the testing (unlabeled target domain) to minimize the discrepancy between the source and target data distributions [15, 16].

We developed a novel, deep domain adaptation model, named Glaucoma Domain Adaptation (GDA), and applied it to three different ocular datasets: the Ocular Hypertension Treatment Study (OHTS) [17], ACRIMA [18], and RIM-ONE [19], for both glaucoma diagnosis and forecasting (prediction). Fundus images in the OHTS were collected from subjects in 22 different centers in the U.S. using different cameras, with considerable variability in image quality, resolution, and characteristics. ACRIMA and RIM-ONE are two publicly available datasets of fundus photographs with annotations that we used for externally validating our model. Although most of the studies of deep learning approaches have been focused on glaucoma screening or diagnosis, we assess the utility of our GDA model for both diagnosing as well as forecasting of glaucoma. We utilized low-rank coding and a weighted source classifier to improve the performance of GDA. Experimental results show the effectiveness of GDA for forecasting and diagnosis of glaucoma disease based on fundus photographs. Our contributions are summarized as follows:

• Creation of a domain adaptation feature learning model via low-rank representation for forecasting and diagnosis of glaucoma disease.

• Mitigation of negative knowledge transfer from the source to the target domain based on a progressive weighting module to improve the accuracy.

• Evaluation of the performance of the proposed model for glaucoma forecasting and diagnosis and compared with state-of-the-art. We show that our model achieved the highest performance compared with numerous existing methods.

The rest of this paper is organized as follows. The related works are reviewed in Section 2. Section 3 introduces our proposed model. Experimental results are given in Section 4. Finally, Section 5 provides a conclusion for this paper.

2. Related Works

2.1. Glaucoma Forecasting and Diagnosis based on OHTS Study

Several studies have been performed for glaucoma diagnosis based on the OHTS dataset but there has been only one published study for forecasting glaucoma based on this dataset.

Thakur et al. [1] developed a CNN architecture based on MobileNetV2 for prediction and glaucoma detection using the OHTS dataset. They used 45,379 OHTS fundus photographs (after excluding ~20% of the images) and achieved AUCs of 0.77 and 0.88 for forecasting glaucoma 4–7 and 1–3 years before glaucoma onset, respectively. They obtained an AUC of ~ 0.95 for glaucoma diagnosis, which was defined on the basis of optic disc or visual field abnormalities.

Two additional studies have used the OHTS dataset for diagnosing glaucoma. Lin et. al. [20] introduced GlaucomaNet, an ensemble deep CNN glaucoma diagnosis model based on DenseNet-201 and ResNet-152 obtained an AUC of ~ 0.90 based on 37,339 fundus photographs from the OHTS dataset. For this study, glaucoma was also defined on the basis of optic disc or visual field abnormalities.

Another study based on 37,339 fundus photographs collected from the OHTS dataset conducted by Fan et. al. [2, 21], utilized a ResNet-50 model to identify glaucoma and obtained AUCs of 0.91, 0.88, and 0.86, respectively for three different definitions of glaucoma that were based 1) solely on optic disc abnormality, 2) a combination of optic disc or visual field abnormality, or 3) solely on visual field abnormalities.

2.2. Deep Domain Adaptation in Glaucoma Research

Machine learning methods used in medical image analysis generally suffer from domain shift problems caused by differences in distributions between source (reference data) and target (query data) domains. To address this problem, domain adaptation (DA) has recently received much attention [22].

Shen et al. [23] proposed a multitask deep learning framework, which utilized an adversarial discriminative domain adaptation approach along with landmark detection to create domain-invariant, interpretable fundus images. They also proposed a semi-supervised DA for image quality assessment, which enhanced the generalized performance between the source and target domains.

Kadambi et al. [24] introduced an adversarial domain adaptation model based on Wasserstein distance for detecting the optic disc-and-cup boundary on fundus images. They exploited the domain adaptation method with Wasserstein distance to calculate cup-to-disc ratio (used to determine glaucoma risk) in an unsupervised manner and calculated the clinical value in the diagnosis of glaucoma.

Xu et al. [25] introduced an unsupervised DA network, named the Minimizing-entropy and Fourier Domain Adaptation network (MeFDA), to decrease the discrepancy among the source and target domains. First, they performed an adversarial optimization method on predicted segmentation entropy maps to decrease the domain shift. Then, they applied direct entropy-minimization to the unlabeled target domain to enhance the validity of prediction segmentation maps. To improve the prediction consistency in the target domain, they augmented the target domain data via the Fourier transform by exchanging the low-frequency section in the target data with corresponding part of the source data.

Liu et al. [26] proposed a multi-scale collaborative adversarial domain adaptation method (CADA) to overcome the underlying domain shift problem for improving the performance of the glaucoma screening convention. They utilized a multi-scale input training manner to overcome the knowledge loss when applying pooling layers in the network and proposed the integration of low-level and high-level features to improve network learning. They applied the adversarial learning method at multiple layers of the network for learning the invariant features for both encoder and decoder phases, simultaneously.

Chen and Wang [27] proposed an unsupervised DA model, named Input and Output Space Unsupervised Domain Adaptation (IOSUDA), to decrease the efficiency degradation in joint optic disc (OD) and optic cup (OC) segmentation in glaucoma disease. Their framework extracts the common content features and the style features of each domain via image translation. They conducted adversarial learning to enhance the similarity of segmentation maps from different domains.

Lei et al. [28] introduced an unsupervised DA method which is based on the image synthesis and feature alignment (ISFA) approach, in order to segment OD and OC on fundus images for assessment of glaucoma disease. They utilized the GAN-based image synthesis strategy along with boundary information for OD and OC to create target-like query data, that can act as the intermediate latent space among source and target domains data, thereby decreasing the domain discrepancy problem.

Liu et al. [29] proposed an adversarial learning-based domain adaptation method (ECSD-Net) to simultaneously solve the joint optic disc and optic cup segmentation task as well as the glaucoma classification task on fundus images from different domains. They combined the efficientdet-D7 with a deep separable convolutional network and used an adversarial based structure to align the distribution between the source and target domains to overcome the domain discrepancy problem.

Haider et al. [30] proposed two networks, the separable linked segmentation network (SLS-Net) and the separable linked segmentation residual network (SLSR-Net), for accurate pixel-wise segmentation of the OC and OD in retinal fundus photographs. SLS-Net and SLSR-Net improve the OC and OD segmentation efficiency via minimization of the spatial information loss. SLSR-Net performs external residual connections for feature empowerment. Both proposed networks comprise a separable convolutional connection to reduce the cost of networks and improve computational efficiency.

Zhou et al. [31] proposed data augmentation-based domain adaptation feature alignment (DAFA) to enhance the out-of-distribution generalization of fundus images for glaucoma detection, which is based on the principle of feature mapping to learn the invariant features and to decrease the effect of data distribution shifts by using maximum mean discrepancy (MMD). They generated two sample views by data augmentation and performed feature alignment between augmented views via semantic representation alignment and latent feature recalibration.

ALGORITHM I.

Low-rank coding optimization procedure

Input: $X_s$, $X_t$, $Y$, $\lambda$

Initialization: $Y_1=Y_2=0$, $Z=0$, $E=0$, $\rho =1.1$, $\epsilon ={10}^{-6}$

Output: $Z$, $E$, ${\mathit{\text{Loss}}}_{lr}$

while converted do

1. Fix the other variables, update $Z$ by solving (7)

2. Fix the other variables, update $J$ by solving (8)

3. Fix the other variables, update $E$ by solving (9)

4. Update the multipliers and parameters: $Y_1$, $Y_2$

5. Check the convergence conditions: ${\Vert X_t-X_{s}Z-E\Vert }_{\infty }<\epsilon$, ${\Vert Z-J\Vert }_{\infty }<\epsilon$

end while

3. OUR PROPOSED GDA MODEL

In this section, we introduce our proposed Glaucoma Domain Adaptation model (GDA) for glaucoma diagnosis and glaucoma forecasting. GDA includes three major structures: general and domain-specific feature extraction, low-rank coding learning, and source classifier loss weighting by considering low-rank loss. In the following, we will discuss these three structures.

3.1. General and Domain-Specific Feature Extraction

As illustrated in Fig. 1, we extract features in two steps. To extract general features between source and target domains, we apply a convolutional neural network (CNN) that projects data from original feature space to the common subspace. We then extract specific features between source and target domains. General features extracted from the common subspace are projected to the specific subspaces. The output of the domain-specific subnetwork has a Softmax classifier for the source domain. We use an entropy classification loss function as provided in (1).

Fig. 1.

Diagram of the GDA that includes a two-stage network for extracting general $\left(G\right)$ and domain-specific $\left(F\right)$ features. The distribution divergence between two domains is reduced via low-rank structure Z. The source classifier has been adapted on the target by weighting method. Domain adaptation between the data of the source domain $\left(X_s\in R^{m\times n_s}\right)$ and the target domain $\left(X_t\in R^{m\times n_t}\right)$ is performed by mapping the data from the original space with dimensions m to a common subspace with dimensions $d$, so that each transferred target sample can be matched with a combination of source samples under low-rank and sparse constraints. The dimensions of low-rank representation coefficient matrix are $Z\in R^{n_s\times n_t}$.

Throughout the network, all $X_s$ (source domain) and $X_t$ (target domain) are first input to CNN model to extract general features $\left(G\right)$, i.e., $G\left(X_S\right)$ and $G\left(X_t\right)$, then the general extracted features are fed to $F$ to extract the domain-specific features, i.e., $F\left(G\left(X_s\right)\right)$ and $F\left(G\left(X_t\right)\right)$. For simplicity, in the rest of this paper, we represent them as $X_s$ and $X_t$.

$$\mathit{\text{Los}}{s}_c=-{\sum }_{i=1}^N{L}_s^i\text{log}\left(S_s^i\right).$$ (1)

Cross entropy loss is minimized based on the probability distribution output $S$ calculated by Softmax with the one-hot-encoded ground truth label $L$. One-hot-encoded implies that in each column-vector only one element is 1 and the remaining are zeroes. Also, we use target domain-specific features to calculate target discriminator loss (2). The classifier trained on the source data is more likely to misclassify target samples that are close to the classification boundary. We thus applied L2 regularization for classifiers’ probabilistic output of target domain data as target discriminator loss to prevent overfitting and enhance the generalization performance of the model.

$$\mathit{\text{Los}}{s}_d={\sum }_{i=1}^N{\Vert S\left(x_t^i\right)-\frac{1}{2}\Vert }^2,$$ (2)

where $S\left(x_t^i\right)$ represents the probabilistic output of the classifier for a given target domain sample $x_t$.

3.2. Low-rank Coding Learning

Let’s $D_s=\left\{X_S,P\left(X_S\right)\right\}$ represent the source domain with data samples $X_sϵR^{d\times n_s}$ and marginal distribution $P\left(X_S\right)$, and $D_t=\{X_t,P\left(X_t\right)\}$ the target domain with data samples $X_t\in R^{d\times n_t}$ and marginal distribution $P\left(X_t\right)$. Domain adaptation is essentially a subgroup of transfer learning when $D_s\ne D_t$ (means that $X_s\ne X_t$ and/or $P\left(X_s\right)\ne P\left(X_t\right)$ while $P\left(Y_s\mid X_s\right)=P\left(Y_t\mid X_t\right)$. The $n_s$ and $n_t$ are number of samples in the source and target domains, respectively. The $ZϵR^{n_s\times n_t}$ is low-rank reconstruction matrix. The dimension of the low-rank space is indicated by $m$. The $EϵR^{m\times n_t}$ is the noise matrix to decrease the effect of noise and negative knowledge. Low-rank optimization is as follows [32]:

$$\underset{ZE}{\text{min}}\parallel Z{\parallel }_{*}+\lambda \parallel E{\parallel }_{1}s.t.X_t=X_{s}Z+E\text{,}$$ (3)

where $\parallel \cdot {\parallel }_{*}$ is the nuclear norm of a matrix [33]. To measure how each target sample is related to source samples, we utilize low-rank coding learning presented in (4) and use the low-rank loss to reduce the discrepancy between source and target domain distributions (ALGORITHM I). The low-rank reconstruction implies that each sample in the target domain can be reconstructed by almost a linear combination of related samples in the source domain, and therefore it can capture more common discriminative information across domains.

So, the low-rank constraint can be applied to coefficient matrix $Z$ for having a block-wise structure [34]. By calculating low-rank loss, we extract the similarity level between each target sample against related source samples. In the following, we optimize the problem shown in (3). To performing that, we introduce a relaxing variable $J$ and change (3) to an equivalent equation as follows:

$$\mathit{\text{Los}}{s}_{lr}=\parallel J{\parallel }_{*}+\lambda \parallel E{\parallel }_{1}s.t.X_t=X_{s}Z+E,Z=J,$$ (4)

where $Z$ is low-rank reconstruction matrix. $E$ is the noise matrix to decrease the effect of noise and unrelated knowledge. $\parallel \cdot {\parallel }_{*}$ is the nuclear norm of a matrix. $J$, $Z$, and $E$ can be optimized by Augmented Lagrange Multiplier (ALM) method [35]. It will be iteratively optimized by updating each variable while the other variables remain constant. The solution of (4) is obtained by minimizing the following function A.

$$A=\parallel J{\parallel }_{*}+\lambda \parallel E{\parallel }_1+\mu ({\Vert X_t-X_{S}Z-E+\frac{Y_1}{\mu }\Vert }_F^2+{\Vert Z-J+\frac{Y_2}{\mu }\Vert }_F^2),$$ (5)

where $Y_1$ and $Y_2$ are Lagrange multipliers and $\mu >0$ is a penalty parameter. The main steps of solving (5) are as follows. All steps have closed-form solutions.

1. Update Z: $Z$ is updated by solving optimization (6).

   $$Z^{*}=\underset{Z}{\text{argmin }}\mu ({\Vert X_t-X_{S}Z-E+\frac{Y_1}{\mu }\Vert }_F^2+{\Vert Z-J+\frac{Y_2}{\mu }\Vert }_F^2).$$ (6)
   To obtain $Z^{*}$, (6) is derived with respect to $Z$ equal to zero. If we assume $G1=X_t-E+\frac{Y_1}{\mu }$ and $G2=J+\frac{Y_2}{\mu }$, the closed form of (6) is as follows:

   $$Z^{*}={\left(\mu X_S^T{X}_S+\mu I\right)}^{-1}\left(G_2-X_S^T{G}_1\right).$$ (7)
2. Update J: To update $J$, the following need to be solved (8).

   $$\begin{gathered} J^{*}=\underset{J}{\text{argmin}}\parallel J{\parallel }_{*}+\mu {\Vert Z-J+\frac{Y_2}{\mu }\Vert }_F^2, \\ {J}^{*}=\frac{{\vartheta }_1}{\mu }\left(Z+\frac{Y_2}{\mu }\right). \end{gathered}$$ (8)

   The singular value decomposition of matrix $X$ is $X=US{V}^T$ with considering singular value $\lambda$, we have ${\vartheta }_{\lambda }\left(X\right)=U{Q}_{\lambda }\left(S\right)V^T$ which is $Q_{\lambda }\left(S_{ij}\right)=sign\left(S_{ij}\right)\text{max}\left(0,\left|S_{ij}-\lambda \right|\right)$.

3. Update E: $E$ is updated by iterating optimization (9).

   $$E^{*}=\underset{E}{\text{argmin }}\lambda \parallel E{\parallel }_1+\mu {\Vert X_t-X_{S}Z-E+\frac{Y_1}{\mu }\Vert }_F^2.$$ (9)
   To solve the above problem, we could use the shrinkage operator:

   $$E^{*}=\mathit{\text{shrink}}\left(X_t-X_{s}Z+\frac{Y_1}{\mu },\frac{\lambda }{\mu }\right).$$ (10)

   Shrinkage operator [36] is defined as: $\mathit{\text{shrink}}\left(x-a\right)=\mathit{\text{signmax}}\left(\left|x\right|-a,0\right)$.

4. Multipliers Y₁, Y₂ and iteration step-size $\rho (\rho >1)$ are updated by (11).

   $$Y_1=Y_1+\mu \left(X_t-X_{s}Z-E\right),Y_2=Y_2+\mu \left(Z-J\right),\mu =\text{min}\left(\rho \mu ,\mu \mathit{\text{max}}\right).$$ (11)

3.3. Weighting the Source Classifier Loss by Considering Low-rank Loss

We utilize a loss function, where a progressive weighting process [37] quantifies the transferability of knowledge so that the source classifier is accurately adapted to classify the target samples. We want to select source samples that are more related to each target sample and increase their contributions to the final model. To do this, we introduce a threshold, in which, if the low-rank loss is more than the threshold, the source classifier loss is weighted to enhance the classification accuracy. We assessed various thresholds such as {0.01, 0.05, 0.1, 0.2, 0.5, 0.8, 1}. The accuracy of GDA first increased and then decreased as threshold changed (as a bell-shaped curve). According to results, we observed a proper trade-off value of 0.1 that achieved the highest performance. This technique mitigates the negative transfer [37, 38], so the learner would not transfer unrelated knowledge from the source domain to the target domain. The weight in transferability weighting scheme is denoted by $W$, which is calculated and normalized by (12). We multiply the weight $W$ with the source classifier $\mathit{\text{Los}}{s}_c$.

$$W=\frac{W}{\frac{1}{B}{\sum }_{i=1}^{B}W\left(x_s^i\right)},$$ (12)

$$\mathit{\text{Los}}{s}_w=W\times \mathit{\text{Los}}{s}_c,$$ (13)

where $B$ is the number of batch size and $w\left(x_s^i\right)=1-p\left(x^i\right)$ is the weight of each source samples. $p\left(x^i\right)$ shows the probability that each sample belongs to the source domain. By weighing the source classifier loss, the ability of source discriminator to adapt on target samples is improved and the contribution of the irrelevant samples of the source domain in the final model is reduced. The total loss of our method includes sum of the weighted source classification loss $\left(\mathit{\text{Los}}{s}_w\right)$, the source classification loss $\left(\mathit{\text{Los}}{s}_c\right)$, the low-rank loss $\left(\mathit{\text{Los}}{s}_{lr}\right)$, and the target discriminator loss $\left(\mathit{\text{Los}}{s}_d\right)$.

$$\mathit{\text{Los}}{s}_{\mathit{\text{total}}}=\mathit{\text{Los}}{s}_c+\mathit{\text{Los}}{s}_w+\lambda \mathit{\text{Los}}{s}_{lr}+\lambda \mathit{\text{Los}}{s}_d.$$ (14)

To mitigate the impact of the noisy activations in the early phases of training, rather than fixing the adaptation factors $\lambda$ and $\gamma$, we used a progressive schedule to gradually increase factors from 0 to 1 as ${\lambda }_i={\gamma }_i=\frac{2}{\text{exp}\left(-\theta \text{i}\right)}-1$, and $\theta =10$ remains constant throughout the investigation [39]. The training of the network is performed by mini-batch stochastic gradient descent (SGD). The main procedure is summarized in ALGORITHM II. More specifically, via minimizing classification loss $\left(\mathit{\text{Los}}{s}_c\right)$, the network was capable of classifying source domain data. More accurately, via weighting the source classifier loss $(\mathit{\text{Los}}{s}_w)$, the network more accurately mitigated the negative transfer, so the learner would not transfer unrelated knowledge from the source domain to the target domain. Via minimizing low-rank loss $\left(\mathit{\text{Los}}{s}_{lr}\right)$, the network learned domain-invariant representations and further aligned domain distributions. Ultimately, via minimizing target discriminator loss $\left(\mathit{\text{Los}}{s}_d\right)$, the network prevented overfitting and enhanced the generalization performance of the model.

4. EXPERIMENTS

4.1. Benchmark Datasets

We evaluated our model based on retinal fundus images collected from three different benchmark glaucoma datasets, including OHTS, ACRIMA, and RIM-ONE. In the OHTS study, we selected two subsets for glaucoma diagnosis and forecasting (prediction). Our results are described in the following subsections.

ALGORITHM II.

The Main Procedure of our method (gda)

Input: $X_s$, $X_t$, $Y$

Initialization: $\mathit{\text{iteration}}=250$, $\mathit{\text{batch\hspace{0.17em}size }}=8$, $\mathit{\text{learning\hspace{0.17em}rate}}=\text{0.01}$, $\mathit{\text{momentum}}=0.9$

Output: $\mathit{\text{Los}}{s}_{\mathit{\text{total}}}$

For $i=1:\mathit{\text{iteration}}$ do

1. Sample $n$ images from source domain $X_s$

2. Sample $n$ images from target domain $X_t$

3. Take source and target samples into general subspace

4. Take source and target general features into domain-specific subspaces

5. Take source domain-specific features into source domain-specific discriminator

6. Use source and target domain-specific features to calculate low-rank loss

7. Weigh the source classifier loss by considering low-rank loss

8. Use target domain-specific features to calculate target discriminator loss

9. Update total loss

end for

4.1.1. The OHTS study

The OHTS study [17] was a prospective and multicenter investigation (22 different centers in the US) that sought to delay or prevent vision loss in ocular hypertensive patients. A total of 66,742 fundus photographs (58,779 single-stereo and 7,963 pair-stereo) from 3272 eyes of 1636 subjects were collected annually throughout the study (~16 years). All patients included in the study initially had both a normal-appearing optic disc and normal visual field at the baseline visit. Details of the OHTS dataset and the process for detecting glaucoma have been described in previous studies [40, 41].

In our research, we utilized OHTS fundus images to determine whether our proposed model can accurately identify glaucomatous eyes before and after glaucoma development (onset of the disease) based on three scenarios: 1) eyes labeled as glaucoma due to optic disc abnormalities regardless of visual field abnormalities, 2) eyes labeled as glaucoma due to optic disc or visual field abnormalities except ones that are glaucoma due to both OD and VF abnormalities at the same time, and 3) eyes labeled as glaucoma due to visual field abnormalities regardless of optic disc abnormalities. We excluded 7,963 pair-stereo images (two fundus photos on a single image) and utilized 58,779 single-stereo images for the experiments. We did not apply any further exclusion criteria to investigate whether the model could work based on real-world images collected under different conditions and settings. We generated two different subsets for glaucoma diagnosis (detection of glaucoma on or after disease onset) and glaucoma forecasting (prediction of glaucoma approximately 1–3 years before disease onset).

The first subset that was used for glaucoma diagnosis included 43,276 fundus images in which 39,385 were from normal eyes and the remaining 3891 images were from eyes with glaucoma (due to optic disc or visual field abnormalities). From 3891 fundus images corresponding to glaucomatous eyes, approximately 29% were labeled as glaucomatous due to optic disc abnormalities only (scenario 1), 22% due to visual field abnormalities only (scenario 3), and 51% due to either optic disc abnormalities only or visual field abnormalities only (scenario 2). Notably, 49% of eyes were glaucomatous due to both optic disc and visual field abnormalities that were not included in our downstream analysis (easiest group with typically more severe cases). The number of selected images for scenarios 1, 2, and 3 were 1128, 1984, and 856, respectively.

The second subset included 40,582 fundus images in which 39,385 fundus images were from normal eyes and 1197 fundus images were from eyes that converted to glaucoma after about 1 to 3 years, based on either optic disc or visual field abnormalities. We used this subset for glaucoma forecasting. The number of selected images for scenarios 1, 2, and 3 were 347, 610, and 263, respectively.

In our experiments, we randomly selected (at patient level) 70% of the data for the source domain and 30% of the data for the target domain [42].

4.1.2. ACRIMA

ACRIMA [18] is a publicly available dataset composed of 705 fundus images collected from 309 normal and 396 glaucomatous eyes. This dataset is part of the ACRIMA project and was obtained from normal and glaucomatous patients with their previous consent and in accordance with the ethical standards of Helsinki. We used ACRIMA as a target domain only in order to validate our model.

4.1.3. RIM-ONE

RIM-ONE [19] is another publicly available dataset that includes 159 stereo fundus images collected from 261 normal and 194 glaucomatous eyes. Three hospitals have contributed to developing this database: Hospital Clínico San Carlos, Hospital Universitario de Canarias, and Hospital Universitario Miguel Servet. In our experiments, we also used this dataset as target domain only.

4.2. Implementation Details

Since training deep CNN models typically requires large training data, we employed a CNN architecture that was pretrained on ImageNet 2012 previously [43]. Our model automatically crops and resizes all images to 224×224×3. The OHTS dataset has an imbalanced number of images in each class, where approximately only 12% of the fundus photographs belong to eyes with glaucoma. To address this problem, we performed data augmentation via random rotations, vertical and horizontal flips, and randomly changed the saturation, hue, and contrast in the training fundus images. After augmentation, during model training, the same number of images were selected from each class in each batch. The coding was performed in Python based on the Pytorch, and all convolutional and pooling layers were fine-tuned based on Pytorch-provided models of ResNet50 [45]. The classifier was trained based on backpropagation. The optimization approach was mini-batch stochastic gradient descent (SGD) with momentum of 0.9 and learning rate ${\eta }_p=\frac{{\eta }_0}{{(1+\alpha p)}^B}$ where $p$ was selected from [0–1] range, ${\eta }_0=0.01$, $\alpha =10$, and $\beta =0.75$. The batch size for minibatch was 8 and the iteration was 250. We trained our model on a GPU machine with Nvidia GeForce RTX 3070.

To estimate the reliability of our proposed model, in the training phase, we split the target domain through a stratified 10-fold cross-validation strategy, whereas we kept one single fold aside for testing and fed the remaining nine folds (without their labels) as unlabeled target domain (along with source domain) into the model. We repeated the training and testing for each fold and then computed the average accuracies as final results.

4.3. Experimental Results

4.3.1. Glaucoma diagnosis on OHTS dataset

Table 1 presents AUCs, sensitivities, and specificities based on different scenarios for glaucoma diagnosis on the OHTS dataset.

Table 1.

Performance of the model for glaucoma diagnosis in terms of AUC, Sensitivity, and Specificity, based on three different scenarios: eyes Labeled as glaucoma due to “only optic disc abnormalities”, “optic disc or visual field abnormalities”, and “only visual field abnormalities” (presented as scenario 1, 2, and 3, respectively). P-values identified based on the method of DeLong. The significance level is p-value < 0.05

Characteristics	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	DeLong P-value
Scenario 1	0.98 (0.96 −0.99)	0.96 (0.94–0.98)	0.90 (0.88–0.92)	Scenario 1~ Scenario 2: P= 0.24
Scenario 2	0.96 (0.94–0.98)	0.94 (0.91–0.96)	0.87 (0.84–0.90)	Scenario 1~ Scenario 3: P= 0.03
Scenario 3	0.93 (0.90–0.95)	0.91 (0.88–0.93)	0.85 (0.82–0.88)	Scenario 2~ Scenario 3: P= 0.12

Glaucoma is typically defined based on the abnormalities in either optic disc or visual field (scenario 2). However, we tested two other scenarios to assess the robustness of our model. As expected, the AUC of the model based on optic disc abnormalities only was significantly better than the AUC of the model based on visual field abnormalities only.

This finding was not unexpected because glaucoma experts report similar findings. Both our model and glaucoma experts appear to more readily identify optic disc abnormality manifested in fundus photographs, compared to identification of abnormalities manifested in visual fields only.

We also evaluated the performance of GDA using area under the receiver operating characteristic curve (AUC) and confusion matrix (Fig. 2). The results showed that the best diagnostic AUC and Accuracy rates were achieved for glaucoma detection based on the first scenario that glaucoma was defined based on the optic disc only (AUC = 0.98, Acc = 0.93); followed by the second scenario that glaucoma was defined based on either optic disc or visual field abnormalities (AUC = 0.96, Acc = 0.91), and lastly, the third scenario that glaucoma was defined based on visual field abnormality only (AUC = 0.93, Acc = 0.88).

Fig. 2.

ROCs and confusion matrices for three scenarios of glaucoma detection. (a) AUCs results of the proposed model for glaucoma detection based on “only optic disc abnormalities”, “optic disc or visual field abnormalities”, and “only visual field abnormalities” were 0.98 (0.96–0.99), 0.96 (0.94–0.98), and 0.93 (0.90–0.95), respectively. (b) the accuracy of glaucoma detection based on only optic disc abnormalities was 0.93, (c) accuracy of glaucoma detection based on optic disc or visual field abnormalities was 0.91, (d) accuracy of glaucoma detection based on only visual field abnormalities was 0.88.

Fig. 2 (a–d) shows the ROC curves and the confusion matrices corresponding to these different scenarios.

Fig. 3 (a–d) show class activation maps (CAMs) representing regions that were more important for the model to make the correct diagnosis. CAMs provide a means to evaluate the clinical relevance of the model. Based on the CAMs, it appears that the model is focused more on the optic disc and peripapillary region, mostly in the inferior hemifield of the retina, which are clinically meaningful, as clinicians also typically evaluate these regions to identify glaucoma.

Fig. 3.

Class activation maps (CAMs) represent regions that were more important for the model to make diagnosis. From left to right (a-d), CAMs correspond to a normal eye, eyes with glaucoma due to only optic disc abnormalities, optic disc or visual field abnormalities, and only visual field abnormalities, respectively.

4.3.2. Glaucoma forecasting based on the OHTS dataset

For glaucoma forecasting, we evaluated the performance of GDA using AUC, specificity, sensitivity, and accuracy on the OHTS dataset based on three different scenarios: 1) optic disc abnormalities regardless of visual field abnormalities, 2) optic disc or visual field abnormalities except ones that are glaucoma due to both optic disc and visual field abnormalities at the same time, and 3) visual field abnormalities regardless of optic disc abnormalities.

Table 2 presents AUCs of the GDA model based on different scenarios for glaucoma forecasting based on the OHTS dataset. As expected, the results showed that the best prediction AUC and Accuracy rates were achieved for glaucoma prediction based on the on the first scenario (glaucoma related to optic disc only) (AUC = 0.90, Acc = 0.82) followed by the glaucoma prediction model based on the on the second scenario (glaucoma related to either optic disc or visual field abnormalities) (AUC = 0.88, Acc = 0.78), then the third scenario (glaucoma related to visual field abnormalities only) (AUC = 0.80, Acc = 0.72). Fig. 4 (a–d) show the ROC curves and the confusion matrices corresponding to these different scenarios.

Table 2.

Performance of the model for glaucoma prediction in terms of AUC, Sensitivity, and Specificity, based on three different scenarios: eyes Labeled as glaucoma due to “only optic disc abnormalities”, “optic disc or visual field abnormalities”, and “only visual field abnormalities” (presented as scenario 1, 2, and 3, respectively). P-values identified Based on the method of DeLong. The Significance level is p-value < 0.05

Characteristics	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	DeLong P-value
Scenario 1	0.90 (0.87 −0.92)	0.83 (0.80–0.86)	0.76 (0.73–0.77)	Scenario 1~ Scenario 2: P= 0.32
Scenario 2	0.88 (0.85–0.90)	0.80 (0.77–0.83)	0.73 (0.70–0.75)	Scenario 1~ Scenario 3: P= 0.01
Scenario 3	0.80 (0.77–0.82)	0.72 (0.70–0.95)	0.64 (0.61–0.66)	Scenario 2~ Scenario 3: P= 0.04

Fig. 4.

ROCs and confusion matrices for three scenarios of glaucoma prediction. (a) AUCs results of the proposed model for glaucoma prediction based on “only optic disc abnormalities”, “optic disc or visual field abnormalities”, and “only visual field abnormalities” were 0.90 (0.87–0.92), 0.88 (0.85–0.90), and 0.80 (0.77–0.82), respectively. (b) the accuracy of glaucoma prediction based on only optic disc abnormalities was 0.82, (c) accuracy of glaucoma detection based on optic disc or visual field abnormalities was 0.78, (d) accuracy of glaucoma detection based on only visual field abnormalities was 0.72.

4.3.3. Comparison with other methods on OHTS dataset

We compared our proposed model with other related methods that also used the OHTS as a benchmark. Thakur et. al. [1] developed a deep CNN model to diagnose and forecast glaucoma based on OHTS fundus images. They obtained AUCs of 0.97 for diagnosis of solely on optic disc abnormality and 0.88 for diagnosis of solely on visual field abnormality. However, they excluded about 20% of the images they deemed ungradable from the total dataset and tested their model on the 15% of the data. In comparison, we only excluded a few percent of the irrelevant images (those with a pair of stereo photos on a single retinal image) and included images with lower qualities and tested our GDA model based on 30% of randomly selected data, and yet were able to obtain slightly better diagnosis and forecasting accuracies. In both models, the testing subsets were selected based on the subject level rather than the eye level to avoid bias. In eye-level analysis, each eye of a patient is analyzed individually however both eyes of the same subjects are typically highly similar thus each eye may carry information from the other eye thus making the learning process biased if two eyes are placed in training and testing subsets. However, all metrics are then calculated at the eye level.

In comparison with another study conducted by Fan et. al. [2], our model obtained significantly higher AUCs with a 7% improvement based on two scenarios for diagnosis of solely on optic disc abnormality and for diagnosis of solely on visual field abnormality. They performed their training at the subject level and selected 20% of data for testing dataset. Table 3 shows the comparison results.

Table 3.

Comparison with other methods on OHTS dataset

	Diagnosis			Prediction
Method	Scenario1	Scenario2	Scenario3	Scenario1	Scenario2	Scenario3
	AUC	AUC	AUC	AUC	AUC	AUC
Thakur et. al. [1]	0.97	-	0.88	-	-	-
Fan et. al. [2]	0.91	-	0.86	-	-	-
GDA	0.98	0.96	0.93	0.90	0.88	0.80

4.3.4. Independent validation of the model based on publicly available datasets

To assure generalizability, we evaluated the performance of our model based on ACRIMA and RIM-ONE datasets (Table 4). In this experiment, we selected the OHTS diagnostic subset as the source and each of the ACRIMA and RIM-ONE datasets as the target domain. The AUCs of our model for detecting glaucoma based on ACRIMA and RIM-ONE datasets were 0.87 and 0.92, respectively (see Fig 5). Findings confirm a reasonably high degree of generalizability reflecting the fact that the model is robust and likely generalizable to unseen datasets.

Table 4.

Performance of the model for glaucoma diagnosis in terms of AUC, Sensitivity, and specificity, when OHTS (Scenario 2) is considered as Source domain and ACRIMA and RIM-ONE are considered as target domain

Source dataset	Target dataset	AUC (95% CI)	Sensitivity	Specificity
OHTS (Scenario 2)	ACRIMA	0.87 (0.83–0.90)	0.85 (0.82–0.88)	0.65 (0.62–0.68)
OHTS (Scenario 2)	RIM-ONE	0.92 (0.89–0.95)	0.89 (0.86–0.92)	0.75 (0.72–0.78)

Fig. 5.

ROCs for scenario 2 of glaucoma detection when the OHTS dataset is as the source domain and each of ACRIMA and RIM-ONE datasets are as the target domain.

4.4. Impact of demographic and ocular parameters on GDA

To investigate whether the GDA model is biased towards gender, age, race, MD, or IOP factors, we performed several supplemental and statistical analyses. We used the Delong’s test [44] for comparing the AUCs of the models and utilized a two-sided p-values of <0.05 for reporting statistical significance. As both eyes may be used in different subsets, we used Generalized Estimating Equation (GEE) to compare those characteristics of the eyes and accounted for both eyes of same subjects in statistical comparisons [45].

We then computed the AUCs of the models based on different genders and races, and different age, MD and IOP ranges. Table 5 shows the AUC of the GDA model based on these factors. As can be seen, the GDA model is not biased to any gender or race. Also, it is not biased towards age or IOP. However, the model was significantly more accurate in detecting those at the later stages of glaucoma (MD < −8 dB) compared to those at the early stages of glaucoma. This is, however, expected as glaucoma is more manifest in those at the later stages of the disease compared to those at the early stages of the disease. As such, the model is robust yet unbiased to demographic factors.

Table 5.

Performance of the model for glaucoma diagnosis in terms of AUC, Sensitivity, and Specificity, based on different genders, ages, races, glaucoma severity levels, and IOP. P-values were calculated based on the GEE model. The significance level is < 0.05

Characteristics	AUC (95% CI)	Sensitivity	Specificity	GEE P-value
Gender				0.46
Female	0.96 (0.94–0.98)	0.94 (0.92–0.96)	0.86 (0.84–0.89)
Male	0.95 (0.93–0.98)	0.93 (0.91–0.96)	0.85 (0.83–0.88)
Age				0.41
< 55 years	0.95 (0.92–0.97)	0.93 (0.90–0.96)	0.86 (0.83–0.90)
≥ 55 years	0.96 (0.94–0.98)	0.95 (0.93–0.97)	0.87 (0.85–0.90)
Race				0.44
Non African-American	0.95 (0.92–0.97)	0.93 (0.90–0.96)	0.85 (0.82–0.88)
African-American	0.96 (0.93–0.98)	0.94 (0.90–0.97)	0.87 (0.84–0.90)
Mean deviation (MD)
≥ 0 dB	0.94 (0.91–0.96)	0.89 (0.85–0.93)	0.82 (0.78–0.86)	mild vs early:0.48
−4 dB ≤ & < 0 dB	0.95 (0.92–0.97)	0.92 (0.89–0.95)	0.84 (0.81–0.87)	moderate vs early: 0.27
−8 dB ≤ & < −4 dB	0.95 (0.93–0.97)	0.93 (0.90–0.96)	0.85 (0.83–0.88)
≤ −8 dB	0.97 (0.94–0.99)	0.95 (0.93–0.98)	0.87 (0.85–0.90)	advanced vs early: 0.04
Intraocular pressure (IOP)				0.38
< 21 mmHg	0.95 (0.92–0.97)	0.91 (0.89–0.94)	0.85 (0.82–0.88)
≥ 21 mmHg	0.96 (0.93–0.99)	0.93 (0.90–0.96)	0.88 (0.85–0.90)

4.5. Justification of findings and clinical validation

While the accuracy of the GDA model was reasonably high, some fundus images were nevertheless misclassified. To identify the reasons for error, we analyzed some of the images that were misclassified. Most of the misclassified eyes in our model were labeled as glaucoma due to abnormalities in visual field only without any obvious sign of glaucomatous optic neuropathy (GON) manifested in fundus images. This can be justified as glaucoma was not identified in such eyes based on clinical evaluations performed by the OHTS imaging reading center as well and they labeled those eyes as glaucoma due to visual field abnormalities only. Additionally, some other misclassified fundus images were of poor quality, including those OHTS images that were scanned from analog fundus photographs. As a result, OHTS represents a dataset with overall low quality. Even so, we did not exclude any fundus photographs due to poor image quality. The results suggest that our novel architecture is more accurate and robust compared to previously developed models.

In a separate experiment, we compared the performance of the GDA model to a human expert. We randomly selected 100 fundus images from the OHTS dataset and compared the diagnostic accuracy of GDA against the diagnostic accuracy of a second-year ophthalmology resident. The diagnosis accuracy of the ophthalmology resident was 79% which was significantly lower than 88% accuracy of the GDA model.

4.6. Ablation Study

As GDA is composed of four loss functions that together minimize the ultimate misclassification error, we performed an ablation study to investigate the influence of each loss function on the GDA performance. Fig. 6 and Fig. 7 show the AUCs of the GDA model based on different ablation studies corresponding to the diagnosis or forecasting tasks. In each scenario, we performed three different ablation studies. Using: 1) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$; 2) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ and $\mathit{\text{Los}}{s}_{lr}$; and 3) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ and $\mathit{\text{Los}}{s}_{lr}$ and $\mathit{\text{Los}}{s}_w$. The baseline structure represents a deep NN based on the ResNet50 architecture. When we added two sub deep networks for extracting general (G) and domain-specific (F) features along with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ to the baseline structure, we obtained AUCs of 0.90, 0.86, and 0.84 based on scenarios 1, 2, and 3 for glaucoma diagnosis, and AUCs of 0.79, 0.76, and 0.71 based on scenarios 1, 2, and 3 for glaucoma forecasting, respectively. By adding the capability of domains alignment and decrease domain discrepancy through $\mathit{\text{Los}}{s}_{lr}$, we obtained improved AUCs of 0.95, 0.93, and 0.91 based on scenarios 1, 2, and 3 for glaucoma diagnosis, and improved ACUs of 0.86, 0.84, and 0.76 based on scenarios 1, 2, and 3 for glaucoma forecasting, respectively. Finally, by adding $\mathit{\text{Los}}{s}_w$ to the previous structure, it forces the source classifier to adapt further to the target samples thus improves the accuracy. In other words, the source samples with more similarity to the target samples are selected and their contributions are boosted in the final model. We obtained improved AUCs of 0.98, 0.96, and 0.93 based on scenarios 1, 2, and 3 for glaucoma diagnosis, and improved AUCs of 0.90, 0.88, and 0.80 based on scenarios 1, 2, and 3 for glaucoma prediction, respectively. As can be seen, the results support our initial idea of integrating these loss functions to improve the robustness and enhance the accuracy of the GDA model.

Fig. 6.

Ablation study based on three glaucoma diagnosis scenarios ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}$: AI baseline structure with cross entropy loss and target discriminator loss functions; ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}+{\text{Loss}}_{\text{lr}}$: AI baseline structure with cross entropy, target discriminator loss, and low-rank loss functions; and ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}+{\text{Loss}}_{\text{lr}}+{\text{Loss}}_{\text{w}}$: AI baseline structure with cross entropy, targe discriminator loss, low-rank, and weighting loss functions.

Fig. 7.

Ablation study based on three glaucoma forecasting scenarios. ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}$: AI baseline structure with cross entropy loss and targe discriminator loss functions; ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}+{\text{Loss}}_{\text{lr}}$: AI baseline structure with cross entropy, target discriminator loss, and low-rank loss functions; and ${\text{Loss}}_{\text{c}}+{\text{Loss}}_{\text{d}}+{\text{Loss}}_{\text{lr}}+{\text{Loss}}_{\text{w}}$: AI baseline structure with cross entropy, target discriminator loss, low-rank, and weighting loss functions.

Further, we calculated the computational complexity for different structures of GDA. We ran each part of our model on a GPU machine with Nvidia GeForce RTX 3070 and recorded the average time of each iteration over 250 iterations. The results were as follows: 1) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ (0.276 s); 2) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ and ${\mathit{\text{Loss}}}_{lr}$ (0.298 s); and 3) baseline structure with $\mathit{\text{Los}}{s}_c$ and $\mathit{\text{Los}}{s}_d$ and $\mathit{\text{Los}}{s}_{lr}$ and $\mathit{\text{Los}}{s}_w$ (0.341 s). Comparing different structures showed that with compromising the computational complexity slightly, the accuracy improved significantly.

5. CONCLUSION

We introduced a novel deep domain adaptation model for diagnosis and forecasting of glaucoma from fundus photographs. The proposed model called GDA includes general and domain specific CNNs and a weighted source classifier to improve accuracy. Domain discrepancy between the source and target domains is minimized using low-rank coding. Experimental results based on three major independent glaucoma datasets showed a high diagnostic accuracy and generalizability. This method may augment glaucoma research and clinical practice by identifying patients with glaucoma and forecasting glaucoma development more accurately.

Like other CNN models, our proposed model can deal with images better than visual fields. Therefore, as expected, our model was more accurate based on fundus images with manifestation of glaucoma due to glaucomatous optic neuropathy compared to glaucoma due to manifestation of visual field abnormalities. Future studies may combine these models with other AI models that can exploit demographic, clinical, ocular, or visual field parameters.

Data availability

Data is available on https://github.com/DM2LL/GDA.