Clinically applicable artificial intelligence algorithm for the diagnosis, evaluation, and monitoring of acute retinal necrosis

Abstract

The prompt detection and proper evaluation of necrotic retinal region are especially important for the diagnosis and treatment of acute retinal necrosis (ARN). The potential application of artificial intelligence (AI) algorithms in these areas of clinical research has not been reported previously. The present study aims to create a computational algorithm for the automated detection and evaluation of retinal necrosis from retinal fundus photographs. A total of 149 wide-angle fundus photographs from 40 eyes of 32 ARN patients were collected, and the U-Net method was used to construct the AI algorithm. Thereby, a novel algorithm based on deep machine learning in detection and evaluation of retinal necrosis was constructed for the first time. This algorithm had an area under the receiver operating curve of 0.92, with 86% sensitivity and 88% specificity in the detection of retinal necrosis. For the purpose of retinal necrosis evaluation, necrotic areas calculated by the AI algorithm were significantly positively correlated with viral load in aqueous humor samples (R ²=0.7444, P<0.0001) and therapeutic response of ARN (R ²=0.999, P<0.0001). Therefore, our AI algorithm has a potential application in the clinical aided diagnosis of ARN, evaluation of ARN severity, and treatment response monitoring.

1 Introduction

Acute retinal necrosis (ARN) is an ocular emergency with devastating consequences, including potential blindness. It can affect both genders at any age (Bonfioli and Eller, 2005; Hillenkamp et al., 2009). Criteria for the definition of its clinical diagnosis include progressive ocular inflammation such as anterior and posterior uveitis, peripheral retinal necrosis, occlusive vasculopathy, optic neuropathy, and disease progression without therapeutic intervention (Holland and Executive Committee of the American Uveitis Society, 1994). Retinal necrosis is the most important feature, however, which initially occurs in the peripheral retina and extends posteriorly in the affected eye (Miserocchi et al., 2019). Resulting necrotizing lesions may cause retinal atrophy and ultimately lead to rhegmatogenous retinal detachment (RRD) (Risseeuw et al., 2019). Therefore, the prompt detection and proper evaluation of associated necrotic regions are crucial for diagnosis and treatment. In clinical practice, the diagnosis of retinal necrosis requires considerable expertise of clinical judgment and decision-making, though the time period for this expertise to provide a timely diagnosis and referral for immediate and adequate treatment of the disease is relatively short.

Computer-aided diagnosis (CAD) technology can improve the consistency of prompt diagnosis, reduce misdiagnosis rate, and decrease physician workload. In computer science, artificial intelligence (AI) is the simulation of human intelligence by computers in order to perform certain elements of human thinking, which contrasts with the natural intelligence displayed by humans and certain other animals. In recent years, breakthroughs were made in computer vision and natural language processing tasks by AI. One efficient application of AI, especially of deep learning, is CAD, which can provide doctors with fast, accurate, and robust analysis of medical data (Puri and Cox, 2018). At the same time, some results have also been achieved through AI in retina fundus image analysis (Gulshan et al., 2016; de Fauw et al., 2018; Kanagasingam et al., 2018; Kermany et al., 2018). The performance of an AI system for diabetic retinopathy deployed in a primary care practice was described by Kanagasingam et al. (2018). Gulshan et al. (2016) attempted to apply deep learning to create an algorithm for the automatic detection of diabetic retinopathy and diabetic macular edema from retinal fundus photographs. In addition, a diagnostic tool based on a deep learning framework for the screening of patients with common treatable blinding retinal diseases was established in the study of Kermany et al. (2018). However, no research is currently targeting the application of AI algorithms for the analysis of the ARN syndrome.

In the present study, we aimed to construct an AI algorithm based on wide-angle fundus images, which could provide a CAD of ARN, and automatically calculate the ratio of retinal areas with necrosis to areas without necrosis. Since ARN is mainly caused by herpes viruses (Culbertson et al., 1982; Schaal et al., 2014), including varicella-zoster virus (VZV), herpes simplex virus (HSV), human cytomegalovirus (HCMV), and Epstein-Barr virus (EBV), the viral DNA load in ocular fluids could also be used to monitor the degree of progress and therapeutic response (Bernheim et al., 2013). We further investigated the potential relationship between viral load and the proportion of retinal necrotic region calculated by AI, and tried to assess the clinical applicability of our AI algorithm for monitoring viral load and response to therapy in ARN.

2 Materials and methods

2.1. Patients’ selection and clinical examinations

Forty eyes of 32 patients (20 males and 12 females, whose ages at disease onset were between 22.0 and 75.0 years) presented at Eye Center, the Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China from 2016 to 2018, who met the diagnostic criteria of ARN (as proposed by the Executive Committee of American Uveitis Society (AUS)), were analyzed in this study. Appropriate informed consents were obtained from all participants in accordance with the Declaration of Helsinki, and the study was approved by the Ethics Committee of Zhejiang University. All patients received intravenous antiviral treatment of 5 mg/kg ganciclovir twice daily or 10 mg/kg acyclovir thrice daily for three weeks followed by oral antiviral treatment (500 mg valacyclovir thrice daily) for three months. When the systemic antiviral treatment alone was not satisfactory, an intravitreal injection of ganciclovir (400 µg) was administered once or twice weekly.

Ophthalmologic examinations including slit-lamp, intraocular pressure measurement, funduscopy, B-scan ultrasound, and wide-angle fundus photograph were performed on all patients. One hundred and forty-nine fundus images were obtained by an Optos ultra-wide field imaging system (Optos Plc Fife, Scotland), which allows for a retinal image of 200° view in a single frame. All images needed to be labeled by two experienced physicians to qualify for AI model training. Specifically, physicians had to circle all lesions along the edges of the image. The AI model considered all areas within each circle as positive, and the remaining areas as negative. The model learned the characteristics of the lesion and automatically circled the lesion area on images. In the current work, the AI model training task was completed by an experienced team of experts from Zhejiang University.

2.2. Algorithm development

We utilized a well-known deep learning method called U-Net (Falk et al., 2019) for retinal necrosis segmentation. The architecture of U-Net is shown in Fig. 1a; it consists of two paths: a contracting path (left side of Fig. 1a) and an expansive path (right side of Fig. 1a).

Fig. 1. Architecture of U-Net (Falk et al., 2019) used in the present study (a) and workflow diagram of the proposed AI system for ARN diagnosis (b). AI, artificial intelligence; ARN, acute retinal necrosis; ReLU, rectified linear unit.

The contracting path, like any typical convolutional network architecture, is composed of eight 3×3 convolution operations, and each pair of convolutions is followed by a rectified linear unit (ReLU) activation layer and a 2×2 max pooling operation for feature activation and abstraction.

The expansive path is composed of four upsampling steps, with each step consisting of an upsampling of the feature map followed by a 2×2 convolution operation that recovers feature details, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3×3 convolution operations followed by a ReLU activation layer. In all 3×3 convolution operations, cropping is necessary due to the loss of border pixels. At the final layer, a 1×1 convolution is used to map each 64-component feature vector to the desired number of classes. In total, the U-Net has 23 convolutional layers and no full-connection layer. In addition, the model will calculate a probability value for each point. If this is greater than 0.5, the pixel area is considered to be a lesion; otherwise, the pixel area is considered normal.

2.3. Quantitative PCR of viruses in aqueous humor

After obtaining consent for aqueous humor sampling, 17 samples from six patients were collected in an ophthalmic surgery room of the eye clinic under sterile conditions from May 2017 to May 2019. Total DNA was extracted from 100 µL of aqueous humor with a DNA Micro kit (QIAamp, USA). Four types of virus (VZV, HSV, HCMV, and EBV) were analyzed by TaqMan quantitative real-time polymerase chain reaction (qPCR) with the respective DNA Fluorescence Diagnostic Kit (Sansure Biotech, China). A total 30 µL reaction volume including 2 µL extracted nucleic acid and 28 µL PCR mixture from the kit was used for each qPCR reaction in the StepOnePlus^™ Real-Time PCR System from Applied Biosystems (Thermofisher, USA). The temperature program for qPCR included a 2-min cycle at 94 °‍C (hot start activation) followed by 45 cycles of amplification (94 °‍C for 15 s and 57 °‍C for 30 s). At the same time, a positive control, a negative control, and a set of four quantification standards were used in each run to evaluate qPCR efficiency and sample quantification.

2.4. Statistical analysis

The performance of the proposed algorithm was evaluated by the area under the receiver operating characteristic (ROC) curve (AUC) generated by plotting sensitivity (true positive rate) versus specificity (true negative rate). Data on viral load were presented as copies/mL and converted to log₁₀ for graphical representation. A Pearson's correlation analysis was performed according to the normal distribution of the ratio of necrotic retinal region in the whole eye area calculated by AI and lg(viral load) (P=0.198 and P=0.172, respectively). Spearman's correlation was conducted to analyze the association between lg(viral load) and the ratio of the area of retinal necrosis accompanied by the therapy. A P-value of less than 0.05 was considered statistically significant. The Statistical Package for the Social Sciences (SPSS) Version 25.0 (SPSS, Inc., Chicago, IL, USA) was used for all statistical analyses in this study.

3 Results

3.1. Characteristics of patients and images

Thirty-two patients, with a mean age of 54.3 (ranging from 22.0 to 75.0) years, participated in the study. Twenty-four patients had only one affected eye (the left in 14 and the right in 10), and the remaining eight cases had developed bilateral ARN. A total of 149 retina fundus images from all 32 patients were collected, with a mean of 4.7 (ranging from 1.0 to 16.0) images per patient.

3.2. AI architecture and performance

The workflow of the whole study is summarized in Fig. 1b. Before model training, the retinal necrosis regions in all 149 images were labeled by two professional physicians, and their annotated areas were selected as necrotic areas. A 5-fold cross-validation strategy was used to achieve our model. In each iteration, 80% of the dataset was employed to train the model, and the remaining 20% was used for model evaluation. Our data augmentation strategy included rotation, translation, and flip. In the training stage, we adopted stochastic gradient descent (SGD) as the optimizer and the initial learning rate was 0.1. We trained our model for 500 epochs, and the learning rate was multiplied by 0.1 every 100 epochs. The weight decay was set to 1×10^-5 to avoid the overfitting problem. Moreover, we used the sigmoid function to get the final score of each point, with a range from 0 to 1 corresponding to the probability of being positive. When the score was under 0.5, we regarded it as a negative point which means that this point was normal, otherwise we considered it a positive point which means that it was a case of retinal necrosis. Binary cross entropy loss was adopted as the objective function, which had previously shown outstanding performance in classification and segmentation problems.

Metrics were computed for each iteration in the testing stage. Finally, our AI algorithm reached 87% accuracy, 86% sensitivity, and 88% specificity on average (AUC=0.92) in identifying ARN regions (Fig. 2a), such as with an image from patient P21 (Fig. 2b), which also indicated that the AI algorithm is of potential value in the clinical aided diagnosis of ARN.

Fig. 2. Performance of the AI algorithm. (a) ROC curve showing AUC values of the AI algorithm. To predict the ARN region, the AI algorithm had an AUC of 0.92. (b) An example of our experimental results, where lighter areas correspond to necrotic retinal areas. AI, artificial intelligence; ARN, acute retinal necrosis; ROC, receiver operating characteristic; AUC, area under the ROC curve.

3.3. Potential clinical application of the AI system

With the aim to further assess the potential clinical application of the AI architecture, we also collected 17 aqueous samples from six patients after obtaining their written informed consent. We identified the causative virus by qPCR technique, and determined the viral load for each sample. Five cases were found with VZV and one case (P32) with HCMV (Table 1). Subsequently, we used linear regression to investigate whether the ratio of necrotic retinal area to total retinal area was correlated with the viral load. This ratio was found to be principally positively correlated with the lg(viral load) (R ²=0.7444, P<0.0001; Fig. 3). Higher viral loads were associated with a larger proportion of necrotic retinal areas. Consequently, the proportion of retinal necrosis regions calculated by AI was likely to be used to monitor viral load in ARN.

Table 1.

Comparison of retinal necrosis ratio as measured by AI and viral load

Patient ID	Virus	Sampling date	Ratio of RN area calculated by AI	lg(viral load)
P27	VZV	2017-05-18	0.0767	5.0227
		2017-06-01	0.0662	3.1541
P28	VZV	2017-06-22	0.3664	5.9750
		2017-06-26	0.3037	6.0828
P29	VZV	2017-07-11	0.3547	6.2746
		2017-07-17	0.1492	5.6854
		2017-07-24	0.1023	5.7645
P30	VZV	2017-10-27	0.4258	6.9699
		2017-10-30	0.5287	6.8593
		2017-11-03	0.3751	6.1734
P31	VZV	2017-09-28	0.3335	6.0445
		2017-10-09	0.1911	4.9741
		2017-10-20	0.1580	4.1979
		2017-10-30	0.1130	3.5411
		2017-11-06	0.0522	3.0496
P32	HCMV	2019-04-01	0.0264	3.5660
		2019-04-08	0.0416	3.2299

AI, artificial intelligence; VZV, varicella zoster virus; HCMV, human cytomegalovirus; RN, retinal necrosis.

Fig. 3. Scatter plot of viral load (log₁₀-transformed) against the ratio of RN area calculated by AI algorithm. AI, artificial intelligence; RN, retinal necrosis.

In the present study, we collected five aqueous samples in total from patient P31 when the first diagnosis was made and after treatment was initiated (Table 1). Viral load was measured about every ten days, and retina fundus imaging was also performed during treatment. Results showed that both virus numbers and necrotic areas were decreasing with the progression of treatment, and this relationship was a basically linear correlation (R ²=0.9999, P<0.0001; Figs. 4a‍–‍4f). Thus, the area calculated by our AI algorithm might be useful to predict the treatment responses of ARN patients.

Fig. 4. Clinical application of AI algorithm for the prediction of treatment responses in one patient (female, 59 years old). (a) Scatter plot of viral load against the ratio of RN area to total eye area accompanied with the treatment process. Images of the ARN-affected eye on 2017-09-28 (b), 2017-10-09 (c), 2017-10-20 (d), 2017-10-30 (e), and 2017-11-06 (f). AI, artificial intelligence; RN, retinal necrosis; ARN, acute retinal necrosis.

4 Discussion

Recently, AI methods have provided promising solutions for the interpretation of medical images and triage definitions, especially for eye-related diseases (Gulshan et al., 2016; de Fauw et al., 2018; Kanagasingam et al., 2018; Kermany et al., 2018). Methods of AI can automatically learn features from the distribution of data. Hence, the application of AI to retina fundus images and other ophthalmology data analyses can qualitatively and quantitatively provide results with great consistency and robustness. With the development of AI, methods with improving accuracy are being proposed, which can be helpful to improve the consistency of diagnosis, reduce the misdiagnosis rate, and decrease physician workload (LeCun et al., 2015). Thus far, no research has targeted the application of AI methods for ARN diagnosis.

ARN is known as a severe necrotizing retinitis, which often leads to blindness. Median delay from the onset of symptoms and signs to correct diagnosis has been reported in previous papers. For example, Sims et al. (2009) even mentioned that the initiation of treatment was delayed by misdiagnosis of sterile uveitis in three cases subject to their study. Such delay is one of the main reasons for the poor recovery of visual function, which frequently happens in rural settings with a lack of access to medical expertise. There is a demand for a simple method to aid clinical diagnosis and evaluation by ophthalmologists. Herein, we present an automated CAD method for retinal necrosis segmentation based on retina fundus images. Our approach has an AUC value of 0.92 in identifying ARN regions and evaluating areas of retinal necrosis. On the one hand, this confirms that AI methods are indeed effective for retina fundus image analysis in ARN. On the other hand, it indicates that our AI approach has a certain application value in clinical aided diagnosis.

Previous studies have demonstrated that the PCR-based analysis of viral DNA allows for a rapid diagnosis of ARN from small quantities of ocular sample (Asano et al., 2004; Calvo et al., 2017), and that viral load has a tight relationship with treatment responses (Bernheim et al., 2013) and proportion of affected retinal regions. Compared to techniques based on viral DNA analysis, AI methods are much simpler in that they are merely based on photographs, are non-invasive and safe for patients. Therefore, we further analyzed the relationship between viral load and the proportion of retinal necrosis areas in two aspects. First, we studied whether the size of necrotic retinal area and viral load are positively correlated among patients. Second, we evaluated the relationship between viral load, namely of VZV, and the area of retinal necrosis during treatment for one patient. Both analyses showed that this correlation was positive, suggesting that the area of retinal necrosis might be used as a marker for the severity of the disease and degree of treatment response. In other words, the AI algorithm might also be applicable for the evaluation and monitoring of ARN.

Nonetheless, our study has several limitations. Firstly, the AI methods have inherent defects, as the number of images to train and test was insufficient. Considering that ARN is a rare disease affecting one in two million people annually, collaboration with multiple eye centers would be necessary to build a larger dataset for future studies. Secondly, we could not perform virus identification in each patient or determine virus quantity in each clinical situation. For this reason, a lot of useful information that could have been associated with necrotic retinal areas might have been lost, which could further decrease the performance of the proposed AI algorithm. Finally, although AI methods for segmentation can reach high levels of accuracy, they cannot always identify boundaries accurately and may produce many false positive cases. Therefore, more powerful AI methods need to be developed by analyzing the relationship between areas of retinal necrosis and viral load, so that necrotic retinal areas predicted by AI will be more accurate and borders of necrotic regions will be clearer; thereby, physicians will get more powerful assistance with ARN diagnosis in clinical practice.

5 Conclusions

In summary, our study is the first demonstration of an AI algorithm for the diagnosis of ARN. The present approach has the potential to perform well in the clinical application of aided ARN diagnosis and the monitoring of treatment effects. Nevertheless, further research with larger datasets and of more comprehensive design is necessary to improve the performance of the deep learning approach in relevant clinical applications.

Author contributions

Ke YAO, Jian WU, and Lei FENG conceived and designed the study. Lei FENG and Junhui SHEN conducted patient recruitment and clinical analysis. Daizhan ZHOU and Chenqi LUO performed the lab examinations and statistical analysis. Wenzhe WANG, Yifei LU, and Jian WU completed artificial intelligence architecture. Lei FENG wrote the manuscript. Ke YAO and Jian WU revised the manuscript. All authors have read and approved the final manuscript. Therefore, all authors have full access to all the data in the study and take responsibility for its integrity and security.

Compliance with ethics guidelines

Lei FENG, Daizhan ZHOU, Chenqi LUO, Junhui SHEN, Wenzhe WANG, Yifei LU, Jian WU, and Ke YAO declare that they have no conflicts of interest.

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Informed consent was obtained from all patients for being included in the study. Additional informed consent was obtained from all patients for whom identifying information is included in this article.