Automated diabetic retinopathy imaging in Indian eyes: A pilot study

Abstract

Aim:

To evaluate the efficacy of an automated retinal image grading system in diabetic retinopathy (DR) screening.

Materials and Methods:

Color fundus images of patients of a DR screening project were analyzed for the purpose of the study. For each eye two set of images were acquired, one centerd on the disk and the other centerd on the macula. All images were processed by automated DR screening software (Retmarker). The results were compared to ophthalmologist grading of the same set of photographs.

Results:

5780 images of 1445 patients were analyzed. Patients were screened into two categories DR or no DR. Image quality was high, medium and low in 71 (4.91%), 1117 (77.30%) and 257 (17.78%) patients respectively. Specificity and sensitivity for detecting DR in the high, medium and low group were (0.59, 0.91); (0.11, 0.95) and (0.93, 0.14).

Conclusion:

Automated retinal image screening system for DR had a high sensitivity in high and medium quality images. Automated DR grading software's hold promise in future screening programs.

India is second only to China in the prevalence of diabetes mellitus. In 2011, the number of diabetics in India were estimated to be approximately 60 million and is projected to rise to 100 million by the year 2030.[1] Over the last 20 years, diabetic retinopathy (DR) has become one of the most common causes of low vision and blindness in India and is currently ranked 6^th amongst the causes of blindness.[2] Due to unavailability of appropriate screening and treatment, especially at the grass root level, a large proportion of diabetics in our country are rendered blind. A statistical survey found that while the national ophthalmologist to population ratio was 0.9:100 000, there are regions in India, which have only 0.3 ophthalmologists per 100 000 population.[3] There is a large gap in the need for DR screening and the available resources. In an attempt to bridge this unmet need, automated DR screening has emerged, with a promise to make available screening facilities to the millions of diabetics who would otherwise go blind due to geographic or economic constraints. Recognizing the significance and benefits of DR screening, some European countries have implemented national screening programs like the DR screening service (Wales),[4] the OPHDIAT^c program (France),[5] and the scottish DR screening program.[6] An increasing number of reports have shown the effectiveness of the automated (yes/no) grading system for DR.[7,8,9,10] There are no reports on automated DR detection in Indian population. We report the use of commercially available Retmarker (Critical health, Coimbra, Portugal) in Indian eyes; and its sensitivity and specificity for detecting DR in Indian population.

Materials and Methods

The present study was carried out at a tertiary care referral eye hospital, in South India. Images were obtained from subjects with diabetes who were part of a population-based cross-sectional and follow-up study: SN-DREAMS, conducted between 2003 and 2006.[11] This study was performed following the principles of the declaration of Helsinki.

Photographic protocol

Fundus examination and clinical grading of diabetic retinopathy

The binocular indirect ophthalmoscope (Keeler Instruments Inc., Pennsylvania, USA) and +20 D lens (Nikon) was used to examine the fundus. The macular area was specifically examined with a +78 D lens (Nikon); to diagnose clinically significant macular edema, as defined by the Early Treatment Diabetic Retinopathy Study (ETDRS).[12,13]

Fundus photography using stereo-pictures

Irrespective of the presence or absence of DR, 45° four-field stereoscopic digital photographs (posterior pole, nasal field, superior and inferior) were taken for all subjects with a Carl Zeiss fundus camera (Visucamlite, Jena, Germany).[13]

Evaluation of photographic grading and quality

The photographic grading and quality were assessed using the VISUPAC digital image archiving system. The “Screenscope” (Berezin Stereo Photography Products, Mission Viejo, CA, USA), a stereo viewer that can be fixed on a computer monitor was used to examine the stereo pairs.

Photographic grading of diabetic retinopathy

The modified classification of DR based on the degrees of retinopathy used by Klein et al.[14] was used; digital photographs were assessed and graded by two independent observers (experienced retinal specialists) in a masked fashion. The photographs were graded against the standard photographs of the ETDRS grading system for severity of retinopathy. In the case of any disparity in the findings, the first specialist consults the second specialist before reaching conclusions. If the two specialists do not agree, the third specialist's opinion will be sought. The grading agreement between first and second specialist was high (k 0.83).[15]

For the purpose of the present study 45° images were retrieved from a cohort of 1445 patients (5780 images). The Retmarker program's protocol included the acquisition of two images per eye and per patient. The photos were taken in the following order (right and left eye, respectively): First photo covering the macular region (field 2); second photo covering the optic disk and nasal region (field 1).[16] These fields were taken from image dataset of the cohort.

Automatic system

The automated grading system, Retmarker, consists of software earmarking micro aneurysms (MAs) and vascular lesions; it includes a co-registration algorithm that allows comparison within the same retinal location between different visits for the same eye. The system generated in a first-step single analysis one of the two possible outputs for the patients, “disease” or “no disease.” Being a deterministic algorithm, its performance is not affected by fatigue, stress, or other factors that may influence a human grader.

Screening approach

Retmarker is patented computer software developed by Critical Health SA (Coimbra, Portugal). It has been certified as a CE mark Class II a medical device. The training of the classifier was done prior to the CE medical device classification, that is, the algorithm (including the classifier) is objective and reproducible, it is not modified or retrained to any individual dataset.[8]

In the automated analysis, the algorithm detects the presence of MAs and vascular lesions in fields 1 and 2. To detect these pathologies, the images are submitted to contrast normalization and enhancement based on principal component analysis. Then, dark objects of a given size are detected and used as candidates.

For each of these candidates, features such as area, shape, intensity distribution were extracted. Next the candidates are classified as true or false. The training of this classifier is done with a dataset in which MAs are marked individually by ophthalmologists to serve as ground truth. The images used for training are not part of the dataset from which the results are generated. The images were graded into low, medium and high quality by an inbuilt software algorithm.[17]

Patients were deemed referable if there were any DR changes. The results between the automated system and the manual grader were compared for study purpose. Data were analyzed using the Statistical Package for Social Sciences (SPSS ver 13, SPSS Inc, Chicago, IL, USA). Sensitivity and specificity data are presented on the performance of the Retmarker system compared with the manual grading as the gold standard.

Results

A total of 1445 patients was screened for DR by both the manual and the automated system and the outcomes of the automated grading system were compared with the standard method of manual screening. Of the 1445 patients screened, 1207 (83.52%) had no DR, 123 (8.51%) had mild nonproliferative DR (NPDR), 80 (5.53%) had moderate NPDR, 17 (1.17%) had severe NPDR and 18 (1.24%) had PDR, as recorded by manual grader. Table 1 shows the distribution of the patients according to the severity of DR. Automated analysis of the acquired images showed that 257 (17.78%) image sets were of low quality while 1117 (77.30%) were of medium quality and 71 (4.91%) image sets were of high quality, however, all the images were gradable by graders [Table 2].

Table 1.

Grading of DR amongst study subjects by expert manual grader

Table 2.

Quality of images used for analysis by software

On assessing the outcomes of the Retmarker according to the image quality it was seen that in the low image quality group, only six patients (2.33%) who needed referral were correctly diagnosed while 37 patients (15.61%) were missed. In the medium image quality group, 162 (14.5%) patients were correctly diagnosed as having DR while 7 (0.62%) were undiagnosed. In the high image quality group 11 (15.49%) patients were rightly diagnosed with DR while only 1 (1.4%) patient was missed. It was noted that while the number of true positives (14.56%) increased and false negatives (0.67%) decreased considerably in the medium and high groups as compared to the low image quality group; the number of false positives (72.47%) also increased. The results of the Retmarker analysis have been summarized in Table 3. The sensitivity of the software increased from 0.14 in the low image quality group to 0.95 and 0.91 in the medium and high image quality groups respectively. The specificity however was quite low that is, 0.11 and 0.59 in the medium and high image quality groups respectively.

Table 3.

Result of software analysis based on image quality

Discussion

In spite of major advances in the management of DR; it is still the leading cause of blindness in the working age population in India and the world. The major obstacle faced when combating this disease is failure to diagnose DR in the early stages; since it has been observed that only 18–35% of diabetics undergo ophthalmologic evaluation.[18]

The prevalence of DR both globally and in India has increased considerably. It is estimated that by the year 2030, India will be home to approximately 100 million diabetics.[19] The prevalence of DR in India as reported by various studies from South India is 12% to 22.7%[20,21] ranging from 18%[15] in the urban population to 10.5%[22] in rural areas. Considering that 75% of India's population resides in rural areas, it is evident that the bulk of the disease burden of DR is contributed by the rural population. This poses a significant problem since the majority of ophthalmologists (70%) in India are concentrated in the urban areas.[23] The national ophthalmologist to population ratio was found to be 0.9:100,000.[3] Abundance of diabetic patients in the rural areas coupled with the scarcity of trained ophthalmologists in these areas translates into a significant number of cases of DR going undiagnosed and eventually resulting in blindness.

Recognizing the dire need for active screening of the population for DR, a number of screening models have been developed in India[24] with an aim of overcoming the various logistic, topographical and socio-economic obstacles faced by the patients, especially the rural population. Due to a limited number of trained ophthalmologists, most of these models have incorporated teleophthalmology into their practice, thereby obviating the need of the physical presence of a trained ophthalmologist at the screening site. Telemedicine includes the assessment and analysis of patient information and interaction by a health professional who is separated temporally and/or spatially from the patient.[25]

In the various teleophthalmology models designed for DR screening, fundus images of the patients that have been acquired as per standardized protocol, along with their clinical profile is transferred to a trained ophthalmologist at a remote site. These images are then graded by the ophthalmologist and those patients in need of treatment are then referred to the higher center for the same. Though telescreening has greatly optimized the use of available manpower and other resources, it is still a daunting task for a designated ophthalmologist to screen thousands of images on a daily basis. Also, it has been noted that about 90% of the cases screened for DR show no evidence of DR and required only regular follow-up. Thus the incorporation of automated screening software that would serve as a primary sieve to filter out the normal images is desirable. In a study by Philip et al.[26] it was seen that the automated disease/no disease grading software used for primary screening reduced the workload of manual graders by 60%.

In our study analogous automated DR screening software, Retmarker was used to analyze the color fundus photos of 1445 patients (5780 images). Similar to previous studies,[26] it was seen that images of 1207 patients (83.52%) had no evidence of DR. Hence incorporation of this grading system as an initial step in DR telescreening would ensure optimal deployment of manpower as well as other resources and improve the cost effectiveness of the screening program.

Another advantage of using an automated grading system is, that unlike manual grading systems, it does not face problems like fatigue and inter or intra observer variability. Automated systems are reliable and repeatable and due to their speed and cost efficiency; one is able to screen a larger population as compared to manual graders.

It has been noted that for any screening test to be effective, it should have a minimum sensitivity of 80%. As we know, a highly sensitive test is useful for ruling out a disease while a highly specific test is useful in ruling in that is, diagnosing a disease. Sensitivity and specificity are seen to be inversely proportional to each other. As a sensitivity increases the specificity decreases and vice versa.[27] Hence, an ideal screening test should have a higher sensitivity as compared to specificity in order to reduce the number of false negatives. In our study, it was noted that 17.7% of the images acquired were of a low quality. We found an overall sensitivity of the Retmarker software to be 79.9%, however when these images were excluded from the study it was seen that the sensitivity of Retmarker increased to 95.9–97.1% for medium and high quality images respectively. Philip et al. have reported a sensitivity of 90.5% and specificity of 67.4% using automated grading system for DR screening of 6722 patients.[26] Other studies have reported higher sensitivities and specificities, but this could be attributed to smaller sample size and no automated quality assessment system.[28,29,30]

Image quality was an issue. This problem was not faced by manual graders while grading the images. In order to mitigate the problem of low quality images reducing the sensitivity of the system, an inbuilt quality control device may be instilled in the system, which informs the technician acquiring the images, in real time, that the acquired images do not meet the predetermined standards. We speculate additionally that there could be another means of increasing the sensitivity of the software in our study. Being the first time such software has been used on Indian population and since this software was developed using Caucasian fundus images as training set, there exist certain racial differences in retinal pigmentation, which may result in a lower sensitivity of this software in Indian eyes. This problem may be solved by providing the software with a substantially sized training set of images of Indian eyes.

Inability of the automated system to detect macular edema, which is vision-threatening condition and needs early referral, is a drawback that needs to be improved upon in the future.

It must be underlined that in order for automated software to be implemented successfully two conditions must be met: The first is following a strict protocol for acquiring the images and having qualified photographers with adequate training. The second is that the acquired images are of sufficient quality for automated processing.

Following the pilot study, there are also in progress activities to develop an efficient algorithm for quality control and an improved version of the algorithm. The results herein published may be revisited in the future using the enhanced algorithms. Standardization of image acquisition, transfer and analysis with respect to Indian conditions are required before implementation of this system in clinical practice.

To conclude this pilot study shows for the 1^st time the efficacy of automated DR imaging in screening studies in Indian population. Larger studies are required to validate its use. Further refinement in the software algorithm is required before it can be incorporated in clinical setting. This holds huge promise in fulfilling the unmet needs of treatment of DR in Indian population.