Application of artificial intelligence in ophthalmology

Abstract

Artificial intelligence is a general term that means to accomplish a task mainly by a computer, with the least human beings participation, and it is widely accepted as the invention of robots. With the development of this new technology, artificial intelligence has been one of the most influential information technology revolutions. We searched these English-language studies relative to ophthalmology published on PubMed and Springer databases. The application of artificial intelligence in ophthalmology mainly concentrates on the diseases with a high incidence, such as diabetic retinopathy, age-related macular degeneration, glaucoma, retinopathy of prematurity, age-related or congenital cataract and few with retinal vein occlusion. According to the above studies, we conclude that the sensitivity of detection and accuracy for proliferative diabetic retinopathy ranged from 75% to 91.7%, for non-proliferative diabetic retinopathy ranged from 75% to 94.7%, for age-related macular degeneration it ranged from 75% to 100%, for retinopathy of prematurity ranged over 95%, for retinal vein occlusion just one study reported ranged over 97%, for glaucoma ranged 63.7% to 93.1%, and for cataract it achieved a more than 70% similarity against clinical grading.

INTRODUCTION

Artificial intelligence (AI) is a general term that means to accomplish a task mainly by a computer, with minimal human beings involved^[1]. In other words, the purpose of AI is to make computers mimic the way of our thinking, and improve our work efficiency in the modern fast-pace life. It has become one of the most influential information technology revolutions^[2]. Great progress has been made in theoretical research and its application as far as we can see. AI is widely accepted as the appearance of many robots in difference fields, especially in bioinformatics. Combined with medicine, some robot-assisted surgery has been conducted successfully. It makes doctor's work more precisely and effectively. Nowadays, AI-assisted medical screening and diagnosis based on images are emerging^[3]–^[5]. As we all hear, melanoma, a skin cancer could be diagnosed with a computer algorithm based on macro images captured by a common camera^[6]. In the field of ophthalmology, especially in the blind-causing diseases, it mainly attributes to medical imaging identification and auxiliary diagnosis.

The application of this technology of AI mainly depends on machine learning^[7], which is represented by mathematical algorithms and models formed through lots of input experience.

SUBJECTS AND METHODS

We searched these English-language studies relative to ophthalmology published on PubMed and Springer databases. Later we gave a classification and statistic. Its application mainly concentrates on the diseases with a high incidence, such as diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, retinopathy of prematurity (ROP), age-related or congenital cataract and few with retinal vein occlusion (RVO).

Principle of Artificial Intelligence

The AI devices mainly fall into two major categories^[8] -the machine learning techniques^[9] and the natural language processing methods. But so far, the former is the auxiliary screening and diagnostic technique what we often talk about^[10].

Machine learning provides techniques or algorithms that can automatically build a model of complex relationships by processing the input available data and generalizing a performance standard^[7]. And it can be briefly described as enabling computers make successful predictions or judgments by repeatedly learning existing representative materials. To be able to form an accurate model, machine learning often requires a large number of training data. And most of them need to be labeled its features in advance by relative authoritative experts. Besides, some other data are used to verify the established algorithm. That means the processes mainly include two parts, training set and validation set. Therefore, an important step is to collect a lot of representative training examples. Some experts mark the easy-identify and distinctive features, and input the computer to make it recognize and remember. It is crucial that the feature selection or extraction requires much experience.

There are mainly two deep learning models, including convolutional neural network (CNN) and massive-training artificial neural network (MTANN)^[11]. They are powerful tools for identifying and classifying images. To our knowledge, CNN and MTANN both have many layers. The major differences are that convolutional operations are underwent within the network in CNN, whereas in MTANN they are outside the network. After an iterative process, the last convolution layer is connected with the whole. What's more, CNN needs much more images than the latter. CNN has been successfully used in many fields, such as, large-scale image classification^[12], scene labeling^[13] and so on.

RESULTS

Some current studies based on machine learning have achieved a satisfactory preliminary outcome. For example, the image identification of non-proliferative diabetic retinopathy (NPDR), proliferative diabetic retinopathy (PDR) and AMD attracts most of the attention. The diagnostic sensitivity for AMD ranged from 75% to 100%. Similarly, the sensitivity of detection and accuracy for PDR ranged from 75% to 91.7% and for NPDR, ranged from 75% to 94.7%. The average rate of diagnosis for these diseases can reach 91.3%^[14]. Also, Ting et al's^[15] study aims to develop and evaluate the deep learning system for DR, AMD and glaucoma, based on the fundus images of multiethnic populations. Compared with professional graders, they conclude that their system can achieve a relative high sensitivity and specificity.

Diabetic Retinopathy and Artificial Intelligence

DR is the leading cause of blindness in the working-age people^[16], which mainly affects the retinal microvasculature, leading to progressive damage^[17]. With more and more people affected, DR is gradually deemed to the global public health problem^[18]. Therefore, the large scale screening of DR is needed urgently to detect potentially threatening changes at early stage which will benefit for treatment and management. As we all know, early intervention is the most cost-effective choice^[19].

The automatic identification of DR has attracted a lot of attention, with studies conducting microaneurysm, hemorrhage, exudation, cotton-wool spot and neovascularization detection, and even further classify stages. Most of them use the fundus images as input. The process can be partly represented by Figure 1^[14]. The computers receive many images labeled with diagnostic lesions, extract their characteristics and finally build a model. And then, it can identify the new input images and give a judgement. Wong et al^[20] propose a method based on microaneurysms and hemorrhages to by a three-layer feed forward neural network to classify the DR stages. Imani et al^[21] form a technique to detected the exudation and blood vessel by morphological component analysis (MCA), and Pavle used the CNN. Yazid et al^[22] put forward that they identified the hard exudation and optic disc based on inverse surface thresholding. Some reports say that they use a Lattice Neural Network with Dendritic Processing (LNNDP) or enhancement techniques to detect blood vessels in retinal images^[23]–^[24]. Akyol et al^[25] detect the optic disc of fundus images automatically by using keypoint detection, texture analysis, and visual dictionary techniques. Niemeijer et al^[26] fast detect the optic disc by a method combined k-nearest neighbour (kNN) and cues. They report that the sensitivity of automatic DR screening ranges from 75%-94.7%, also the specificity, accuracy is comparable and promising.

Figure 1. A fundus image is submitted to locate anatomic structures and lesions followed by feature extraction and analysis. The features are an index for searching the library to compare with similar images from database. It can also combine the patient's clinical metadata.

Furthermore, there will be a few studies involved with multimodal data to verify a disease more precisely. For instance, combining macular optical coherence tomography (OCT) with fundus image indentify macular edema, which is the sign of timely treatment. After all, a study has reported an algorithm can detect and quantify subretinal or intraretinal fluid based on OCT images, just described as Figure 2^[27].

Figure 2. Illustration of the automated detection of macular fluid in OCT.

The intraretinal cystoid fluid is marked in green, subretinal fluid is marked blue. AMD: Age-related macular degeneration; DME: Diabetic macular edema; RVO: Retinal vein occlusion.

Apart from the above automatic detection and identification of DR, the study of the evaluation of deep learning models for DR grades. They reported the errors of deep learning models mainly concentrated on missing the microaneurysm and artifacts. For the moderate or worse DR, the sensitivity of deep learning models is about 97.1%, compared with the ophthalmologists' 83.3%. Maybe the quality of input images is responsible for the minimal lesions missing, they think^[28].

Age-related Macular Degeneration and Artificial Intelligence

AMD is a chronic and irreversible macular disease characterized by drusen, retinal pigment changes, choroidal neovascularization, hemorrhage, exudation and even geographic atrophy^[29]. It is one of the leading causes of central vision loss in people aged over 50^[30]. With the social population aging and the severity of this disease, it's necessary to perform AMD screening regularly. Automatic AMD diagnosis may obviously reduce the work load of clinicians and improve efficiency.

Many studies have reported their preliminary results. Most of them use fundus images as input original materials, and extract features of early, intermediate and late AMD to distinguish from the healthy images^[31]. They can obtain a sensitivity ranging from 87% to 100%, also with a relatively high accuracy^[32]. They think taking fundus photo as input is cheaper than OCT examination. But also, there exist researches combined spectral domain OCT with deep learning about AMD, including the macular fluid quantity of neovascular AMD (nAMD) just like Figure 2 and the retinal layers segmentation of dry AMD like Figure 3^[33]. After an iteration training, the training and validation accuracy are both 100%^[34]. They believe that other macular diseases will obtain the same effective results.

Figure 3. Outline of the algorithm to segment the retinal layers of dry AMD.

As we all know, intravitreal injection of anti-VEGF drugs is the first-line therapy for nAMD^[35] and the follow-up observation is also very important. Bogunovic et al^[36] utilize an algorithm to observe the treatment responders using OCT images. Some researchers combine the machine learning with OCT images to observe and predict the possibility of retreatment^[37]. The model they built achieves a comparable performance for predicting the low and almost 50% better performance in predicting the high retreatment requires.

Retinal Vein Occlusion and Artificial Intelligence

RVO has an estimated prevalence ranging from 0.3% to 2.1%^[38]–^[40] in different individuals, which is one of the most common blind-causing diseases, ranking after DR^[41]. We think, the direct reason of RVO may be that sclerotic retinal artery compress the retinal vein and block the blood return of terminal arborizations. Further, it causes superficial hemorrhage, exudation, and retinal edema. If any lesion involves macular, it will lead to vision acuity decreased significantly, or even blindness. Its risk factors mainly are people with old age and vascular sclerosis^[42]–^[44], such as hypertension, arteriosclerosis or cardiovascular disease. Thus, the early diagnosis of RVO is crucial for vision recovery.

Automatic diagnosis will benefit both patients and ophthalmologists, if it is widely used. At present, the machine learning in RVO is relatively rare. A team reported that they utilized CNN combined with patch-based and image-based vote methods to recognize the fundus image of branch retinal vein occlusion automatically. They received a high accuracy over 97%^[45]. It's encouraging for the following researches.

Retinopathy of Prematurity and Artificial Intelligence

ROP is a leading cause of childhood blindness all over the world^[46]–^[47] and it is largely treatable with appropriate and timely diagnosis. Clinical studies have shown that ROP with plus disease or retinopathy in zone one stage 3 even without plus disease requires timely treatment to prevent blindness, and infants with pre-plus disease require close observation^[48]. Repeated screening and follow-up of ROP will consume a lot of manpower and energy. So the application of AI in ROP screening may improve the efficiency of ROP care.

Many studies have tried the automatic identification of ROP. Most of them focused on two-level classification (plus or not plus disease)^[49]–^[52]. They achieved a promising result. An report says that they could distinguish the plus disease with a 95% accuracy, which is comparable to experts' diagnosis, much more precise than non-experts^[53]. And Ataer-Cansizoglu et al's^[54] study took advantages of tortuosity and dilation features from arteries and veins to distinguish not plus or pre-plus or plus disease. They classify the ROP more specifically and benefitial for the treatment.

Anterior Segment Diseases and Artificial Intelligence

Maybe, cataract and glaucoma are very common diseases in ophthalmology^[55]–^[56]. It is not surprising that there are some reports about the application of machine learning in anterior segment diseases^[57]–^[61]. Cataracts are a clouding of the lens and the leading cause of blindness all over the world^[55]. The automatic recognition will be cost-effective.

Gao et al^[57] have reported that they proposed a system automatically grade the severity of nuclear cataracts by slit-lamp images. First, they find the lens region of interest and then CNN filters randomly select image patches generating local representations by an iteration process with random weights. Their system achieved a more than 70% similarity against clinical grading. Other like the research of Liu et al^[58], they mainly focus on the identification of pediatric cataracts. They achieve an exceptional accuracy and sensitivity in lens classification and density. Also, it can automatically grade a cataract by lens OCT^[59].

Glaucoma is a disease that mainly damages the optic nerve, which can cause irreversible blindness^[56],^[60]. Although glaucoma may not be cured, the processing can be slow down by reasonable treatment^[61]. Thus, early detection of glaucoma is highly needed. The detection of glaucoma mainly depends on the intraocular pressure, thickness of retinal nerve fiber, optic nerve and visual field examination^[62]–^[63].

Omodaka et al^[64] developed a machine learning algorithm to classify the optic disc of open-angle glaucoma and reached a accuracy of 87.8%. Their algorithm based on the quantitative parameters mainly from the optic disc OCT examination. Many studies have tried to apply the machine learning in glaucoma identification. The machine usually assesses the cup disc ratio^[65]–^[66] in the fundus images, the visual field^[67] or the thickness of retinal nerve fiber examined by OCT^[68]. The accuracy of early diagnosis ranges from 63.7% to 93.1% depending on the input images.

DISCUSSION

AI-assisted automated screening and diagnosis of the common diseases in ophthalmology may eventually help maximize the doctors' role at the clinic. Outside the clinic, AI platforms offer the patients more medical opportunities and reduce obstacles to access for an eye care where an ophthalmologist is not available. To some extent, new technologies based on AI may reduce social inequalities^[69]. Looking further into the future, AI-assisted system shows the potential to relieve the overburdened healthcare system's problems.

In general, the process of automatically detect a disease mainly include three steps^[11],^[70]. Firstly, it's necessary to collect a large amount of images, and relative experts have to label the characteristic lesions. It is fundamental but very crucial. Secondly, computers extract the features of a disease through a particular program based on the input of marked images. Finally, a given image can be distinguished from other kind of disease by statistical feature of target lesions.

According to these studies, some algorithms have been preliminarily formed, such as DR, ROP, AMD, RVO, glaucoma, cataract and so on. However, with so many present reports, there is seldom one realized a 100% accuracy and sensitivity. That is to say, not every image can be identified precisely or not be missed. Not only does it depend on the computer technique, but the quality of input images^[71]–^[72]. The main factors caused poor quality of posterior or anterior segment images may be the patient's head or eyeball movement, undilated pupil, frequent blinking, opaque refractive medium and poor fixation^[73]–^[74]. Besides, the marking process by experts is also quite important. It's the foundation of computer learning. Thus, the annotators must be trained for a uniform standard.

Besides that, there may exist some other limitations about deep learning^[11],^[75]. First, forming an algorithm needs a lot of computational cost and training experience. That means AI may be just useful for the diseases with a high morbidity. For rare diseases, it may not be available. Second, the computer recognizes a structure or a feature mechanically, so AI could not completely identify a disease separated from our intervention. A small portion of feature and variation that look like unusual will be missed. We infer that AI can pick out the majority of people with a kind of disease, not all of them. Third, to some extent, this work is complicated. The characteristics of a disease and parameters of an algorithm differ from tasks to tasks. Finally, if the relationship between input and expected output materials is complex, the machine will probably not build a model. What's more important is that it may cause a mistake. Nguyen et al^[76] described the process how the neural networks lead to a wrong classification.

From this perspective, AI can really efficiently conduct a task, but a certain degree of human intervention is essential during the process.

In conclusion, AI has been widely studied in ophthalmological image processing, mainly based on the fundus photographs. Indeed, it achieves a promising accuracy comparable with clinical experts. However, more efforts should be made to explore the neural network to assist our work.