Automated detection of diabetic retinopathy in retinal images

Abstract

Diabetic retinopathy (DR) is a disease with an increasing prevalence and the main cause of blindness among working-age population. The risk of severe vision loss can be significantly reduced by timely diagnosis and treatment. Systematic screening for DR has been identified as a cost-effective way to save health services resources. Automatic retinal image analysis is emerging as an important screening tool for early DR detection, which can reduce the workload associated to manual grading as well as save diagnosis costs and time. Many research efforts in the last years have been devoted to developing automatic tools to help in the detection and evaluation of DR lesions. However, there is a large variability in the databases and evaluation criteria used in the literature, which hampers a direct comparison of the different studies. This work is aimed at summarizing the results of the available algorithms for the detection and classification of DR pathology. A detailed literature search was conducted using PubMed. Selected relevant studies in the last 10 years were scrutinized and included in the review. Furthermore, we will try to give an overview of the available commercial software for automatic retinal image analysis.

Diabetic retinopathy (DR) is the leading cause of blindness in the working-age population.[1] Screening for DR and monitoring disease progression, especially in the early asymptomatic stages, is effective for preventing visual loss and reducing costs for health systems.[2] Most screening programs use nonmydriatic digital color fundus cameras to acquire color photographs of the retina.[3] These photographs are then examined for the presence of lesions indicative of DR, including microaneurysms (MAs), hemorrhages (HEMs), exudates (EXs), and cotton wool spots (CWSs).[4] In any DR screening program, about two-third of patients have no retinopathy.[2] The application of automated image analysis to digital fundus images may reduce the workload and costs by minimizing the number of photographs that need to be manually graded.[5]

Many studies can be found in the literature regarding digital image processing for DR. Most algorithms comprise several steps. First, a preprocessing step is carried out to attenuate image variation by normalizing the original retinal image.[6] Second, anatomical components such as the optic disk (OD) and vessels are removed.[7] Finally, only those remaining pathological features of DR are retained for subsequent classification. This review gives an overview of the available algorithms for DR feature extraction and the automatic retinal image analysis systems based on the aforementioned algorithms.

Materials

The methodological quality of published articles was evaluated, and the following inclusion criteria were defined. Only studies published in English and indexed in PubMed in the last 10 years (2005–2015) were considered. In addition, the results of these studies must be presented using mean sensitivity (SE), mean specificity (SP), or area under the Retinopathy Online Challenge (ROC) curve. Research works using image modalities different from color retinal images or aimed at pathologies different from DR were dismissed.

Methods and results of literature search for EXs segmentation algorithms, red lesions (RLs) segmentation algorithms, and DR screening systems are presented in the following section.

Segmentation of exudates

EXs are lipoprotein intraretinal deposits due to vascular leakage.[8] They appear in retinal images as yellowish lesions with well-defined edges. Their shape, size, brightness, and location vary among different patients.[9] When clusters of EXs are located in the macular region, they are indicative of macular edema (ME), which is the main cause of visual loss in DR patients. For this reason, many researchers introduced the idea of a coordinate system based on the location of the fovea to determine DR grading.[10]

Different techniques have been proposed for EXs detection. In Table 1, information regarding the results of these methods and the databases used in each study is summarized. They can be divided into four categories.[11]

Table 1.

Performance of exudates segmentation methods

Region growing methods

With these techniques, images are segmented using the spatial contiguity of gray levels. In the method described by Sinthanayothin et al.[12] adjacent pixels were considered as belonging to the same region if they had a similar gray level or color.[12]

Thresholding methods

With these methods, EXs identification was based on a global or adaptive gray level analysis. As EXs are mainly characterized by their color, Sánchez et al.[13] employed color features to define a feature space. They proposed a modification of the RGB model and used the intensity of various pixels in the new color space to create their training set.[13,14] Other authors divided the image into homogeneous regions and applied an adaptive thresholding method to each region.[15,16]

Mathematical morphology methods

The algorithms based on these methods employed morphological operators to detect structures with defined shapes. Different morphological operators were used for EXs detection in the work of Sopharak et al.[17] Zhang et al.[18] proposed a two-scale segmentation method. To detect large EXs, authors performed a morphological reconstruction followed by a filtering and thresholding operation. Then, authors applied a top-hat operator to the green channel of the original image to recover small EXs.[18]

Classification methods

These studies employed machine learning approaches to separate EX from non-EX regions, including additional types of bright lesions (BLs), such as drusen and CWSs. Although a classification stage was also part of many of the previous studies, we have included in this category only those studies for which classification was the main step.

In Osareh et al.,[19] images were segmented using a combination of color representation in the Luv color space and an efficient coarse to fine segmentation stage based on fuzzy c-means clustering.[19] A similar approach was proposed by García et al.,[20] who combined global and adaptive histogram thresholding methods to coarsely segment bright image regions. Finally, a set of features was extracted from each region and used to assess the performance of three neural network (NN) classifiers: Multilayer perceptron (MLP), radial basis function (RBF), and support vector machine (SVM).[20]

A multi-scale morphological process for candidate EX detection was proposed by Fleming et al.[9] A SVM was subsequently used to classify candidate regions as EX, drusen or background based on their local properties.[9] In the study by Niemeijer et al.,[21] each pixel was assigned a probability of being an EX pixel, resulting in a lesion probability map for each image. Pixels with a high probability were grouped into probable lesion pixel clusters. Then, each pixel was classified as true EX or as non-EX depending on the cluster characteristics. Finally, a BL was classified into EX, CWSs or drusen by means of a k-NN classifier and a linear discriminant classifier.[21]

Segmentation of red lesions (microaneurysms and hemorrhages)

MAs are small saccular bulges in the walls of retinal capillary vessels.[22] In color fundus images, MAs appear like round red dots with a diameter ranging from 10 to 100 µm. MAs are difficult to distinguish from dot-HEMs, which are a little bigger.[23] MAs are normally the first retinal lesions that appear in DR and their number has a direct relationship to DR severity.[24]

Several approaches have been proposed for MAs segmentation through color image analysis. In Table 2, information regarding results of these methods and the databases used in each study is summarized. The methods for RL detection can be also divided into four categories.[25]

Table 2.

Performance of red lesions segmentation methods

Region growing methods

Fleming et al.[22] evaluated an algorithm where region growing was performed on a watershed gradient image, to identify candidate RL regions.[22] An interesting MA detection algorithm was developed at the University of Waikato and validated by Jelinek et al.[26] It was an automated MA detector inspired by the detectors developed by Cree et al.[27] and Spencer.[24] A top-hat transformation was first used to discriminate between circular, nonconnected RL and the elongated vasculature. Candidate lesions were then segmented by means of a region growing algorithm.

Mathematical morphology methods

A polynomial contrast enhancement operation, based on morphological reconstruction methods, was used by Walter et al.[28] to detect MAs and to discriminate between MAs and vessels.[29]

Wavelet-based methods

The method proposed by Quellec et al.[30] was based on template matching using the wavelet transform. Images were descomposed in subbands, each subband having complementary information to describe MAs.

Hybrid methods

A hybrid RL segmentation algorithm was developed by Niemeijer et al.[31] The system combined the candidates detected using a mathematical morphology based algorithm with the candidates of a pixel classification based system.[31] An approach based on multi-scale correlation filtering was evaluated by Zhang et al.[32] For candidate detection, authors calculated the correlation between pixel intensity distributions throughout the image and a Gaussian model of MAs using a sliding window technique.[32] A different approach was based on calculating cross-section profiles along multiple orientations to construct a multi-directional height map.[33]

Another method based on feature classification was proposed by García et al.[34] A set of features was extracted from image regions, and a feature selection algorithm was applied in order to choose the most adequate feature subset for RL detection. Four NN-based classifiers were used to obtain the final segmentation: MLP, RBF, SVM, and a combination of these three NNs using a majority voting schema.[34,35] Sanchez et al.[36] used a three-class Gaussian mixture-model-based on the assumption that each pixel belonged to one of the three classes: Background, foreground (vessels, lesions, and OD), and outlier.[36] The method developed by Mizutani et al. used a modified double ring filter, which extracted MAs along with blood vessels. This method was designed to detect areas of the image in which the average pixel value was lower (inner circle) than the average pixel value in the area surrounding it (outer circle).[37,38]

Due to the numerous MA detection methods published, an international MA detection competition, the retinopathy online challenge (ROC), was created to compare the results of different methods.[4] The dataset used for the competition consisted of 50 training images with available reference standard and 50 test images for which expert annotations were not provided. This permitted a fair comparison between algorithms proposed by different groups. The results of five different methods such as Valladolid,[36] Waikato,[26] Latim,[30] OkMedical,[32] and Fujita Lab[37] were presented by five different teams of researchers. The results of these five MA detection methods were evaluated using the ROC database.[4]

Diabetic retinopathy screening systems

The previously mentioned studies and the related algorithms have enabled different research groups to develop computer-aided diagnosis (CAD) systems for DR screening. Information regarding results of these methods is shown in Table 3. DR severity levels are characterized by the number and type of retinal lesions that appear in the image, as well as by the retinal area in which these lesions appear. Different authors have proposed several DR severity scales to automatically determine the stage of DR in a patient.

Table 3.

Comparison of automatic diabetic retinopathy screening systems

The work developed by Singalavanija et al.[39] tried to differentiate between normal and DR fundus. Although authors detected DR lesions with a good SE and SP, their DR screening system was not sensitive enough to detect early stages of nonproliferative DR (NPDR).[39] In the same way, Dupas et al.[40] determined the severity of DR. Grade 0 was established when no RLs were detected. Besides, Grades 1, 2, and 3 were defined according to the number and type of RLs detected in a retinal image. Authors also evaluated the risk of ME in a patient according to the distance between EXs and the fovea.[40] The automated system proposed by Tang et al.[41] separated normal retinal images from unhealthy images. Images with different quality and resolution were used. No DR severity grading was reported in this study.[41]

A different DR severity grading method was proposed by Usman Akram et al.,[42] based on the type (RLs and EXs) and number of lesions detected. Only images without lesions were considered normal images.[42] In other studies,[43] authors proposed a four different severity grades. Level R₀(no DR) corresponded to the case where no RLs were found in an image. Besides, levels R₁, R₂, and R₃(DR images) corresponded, respectively, to the cases where a small, medium and large number of RLs appeared in an image. Authors evaluated the accuracy of their algorithm in distinguishing normal (R₀) from pathological (R₁, R₂, R₃) images.[43] Roychowdhury et al.[44] designed a system based on machine learning techniques. Images were classified as with or without DR according to the number of RLs detected.

Several automatic retinal image analysis systems have already become commercially available.[45] These include the Retinalyze System^®, which combines the ability of RL detection and image quality control to identify patients with DR and separate them from patients with no signs of DR.[46,47] In the same way, iGradingM^® performs “disease/no disease” grading for DR. This software combines image quality assessment algorithms with MA detection methods.[48] SE above 90% for referable retinopathy was achieved[49,50] and showed a manual grading workload reduction of 36.3%.[51] This system was later tested[52] using two-field photographs. In this study, authors found that the inclusion of a second image (disk centered field) did not improve the results. Besides, they also established that including the detection of other types of lesions in the screening system resulted to similar SE but also in a higher number of false positives when compared to the case in which only MAs were considered.[52]

Another available system is IDx-DR^®.[53,54,55] It uses several algorithms developed at the University of Iowa for DR lesions detection, such as MAs and HEMs,[31,30] EXs and CWSs.[21] IDx-DR^® also includes algorithms for the detection of other types of DR signs, such as neovascularization.[23] The system was validated on a database of 1748 fovea-centered images.[53] The aim of the study was to validate the system in referable DR detection. Referable DR was defined as more than mild NPDR and/or ME. Using IDx-DR^®, authors found that the prevalence of referable DR was 21.7%.[53]

The software RetmarkerDR^®, developed at the University of Coimbra, should also be mentioned. It is based on combining image quality control with RL detection.[56] The system was able to separate images with no signs of DR or with no evolution of DR compared to previous screening visits, from those with signs of DR pathology or evolution.[56] The system showed a potential reduction of 48.42% in the workload of human graders.[57] Finally, Telemedical Retinal Image Analysis and Diagnosis Network^® is a web-based service in which the quality of retinal images is automatically evaluated.[58,59]

These systems have been successfully applied in DR screening scenarios to identify the presence of DR or referable DR. However, to the best of our knowledge, they are not able to identify the high-risk DR or the presence of DME yet.[45]

Discussion

Early DR detection is important to slow down disease progression and avoid severe vision loss in diabetic patients. Regular DR screening is paramount to ensure timely diagnosis and treatment. However, the interpretation and grading of fundus images for this task is actually a manual process. This is a time-consuming approach, which is also subject to inter-observer variability. For this reason, automatic methods to detect DR-related lesions, as well as the development of CAD systems for DR can be a reliable option to cut down DR screening costs, to reduce the workload of ophthalmologists and to improve attention to diabetic patients. In this regard, the British Diabetic Association (BDA) estimates that the rates of any screening program for DR should reach SE >80% and SP >95%.[60] This issue should be considered when comparing the different alternatives for DR lesions detection and screening.[61]

The algorithms for DR lesions detection included in this review were very heterogeneous. The validation methods and test databases of the studies were not uniform or standardized. Therefore, it was not possible to make a direct comparison of their performance. However, some results should be underlined. In the case of EXs, nine studies complied with the inclusion criteria. Most of these studies achieved the BDA figures. As shown in Table 1, the highest lesion-based SE and SP were obtained with the method proposed by Jaafar et al.[15] and the highest image-based SE and SP were obtained by Sánchez et al.[14] However, it should be mentioned that the results of the studies in Table 1 are not directly comparable due to the lack of common measurement criteria and evaluation databases. For example, the studies by Fleming et al.[9] and Niemeijer et al.[21] used a larger database. Besides, these two studies were remarkable because they included the detection and differentiation of several types of BLs.

In the case of RLs, 12 studies met the inclusion criteria for this review. The SE and SP values, in this case, are generally below the figures for EXs detection. This indicates that RLs detection is more challenging. However, the results obtained in some studies [Table 2] were only slightly below the BDA values. It should be noted that not only SE and SP figures are important; the number of images employed in each study must be taken into account. In the case of Fleming et al.[22] and Jelinek et al.[26] a higher number of images were used than in others studies with better results. It should also be mentioned that results of the ROC competition were not measured in terms of SE and SP.[4] Thus, we could not evaluate whether the participants met the requirements of the BDA. However, it is noteworthy that the methods included in the competition could be directly compared using a common database and evaluation criteria.

The lesion detection algorithms have also allowed the development of CAD systems for DR screening. Several of the studies included in Table 3 focus on separating DR and healthy cases.[39,41,44,46,48,56] Other authors also attempt to make a first DR severity grading by distinguishing referable and nonreferable DR cases.[40,42,43,53] This means that there are computationally efficient algorithms that could allow the detection of derivable DR. Moreover, most of commercially available DR screening systems reached 80% SE in detecting DR cases, although none of them achieved the SP values recommended by the BDA. The inclusion of a DR severity grading stage poses additional complexity to these systems since DR severity grades were not standardized and nonuniform severity scales were used.

Although the results of the developed algorithms are promising, challenges still remain. Further work is necessary to improve the proposed CAD systems so they can efficiently reduce the workload of ophthalmologists. First, the proposed methods should be tested on larger databases to ensure that they are capable of preventing visual loss in DR patients in a cost-effective way. Although there are some publicly available databases designed for automatic retinal image analysis, algorithms should be tested in a larger datasets representative from screening scenarios. In addition, inter- and intra-observer variability should be addressed in studies related to DR lesions detection or DR screening software development. Finally, DR severity grading systems should be consistent with clinically approved DR severity scales and thus, consider the different signs of DR. Despite these difficulties; several research groups are working toward the improvement and validation of CAD systems to efficiently diagnose DR and determine the DR severity grade in a patient. The final goal would be to develop an automatic system for DR screening with enough accuracy to be incorporated in the daily clinical practice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.