Abstract
Purpose
To evaluate the accuracy of different viewing monitors for image reading and grading of diabetic retinopathy (DR).
Design
Single-centre, experimental case series—evaluation of reading devices for DR screening.
Method
A total of 100 sets of three-field (optic disc, macula, and temporal views) colour retinal still images (50 normal and 50 with DR) captured by FF 450 plus (Carl Zeiss) were interpreted on 27-inch iMac, 15-inch MacBook Pro, and 9.7-inch iPad. All images were interpreted by a retinal specialist and a medical officer. We calculated the sensitivity and specificity of 15-inch MacBook Pro and 9.7-inch iPad in detection of DR signs and grades with reference to the reading outcomes obtained using a 27-inch iMac reading monitor.
Results
In detection of any grade of DR, the 15-inch MacBook Pro had sensitivity and specificity of 96% (95% confidence interval (CI): 85.1–99.3) and 96% (95% CI: 85.1–99.3), respectively, for retinal specialist and 91.5% (95% CI: 78.7–97.2) and 94.3% (95% CI: 83.3–98.5), respectively, for medical officer, whereas for 9.7-inch iPad, they were 91.8% (95% CI: 79.5–97.4) and 94.1% (95% CI: 82.8–98.5), respectively, for retinal specialist and 91.3% (95% CI: 78.3–97.1) and 92.6% (95% CI: 81.3–97.6), respectively, for medical officer.
Conclusion
The 15-inch MacBook Pro and 9.7-inch iPad had excellent sensitivity and specificity in detecting DR and hence, both screen sizes can be utilized to effectively interpret colour retinal still images for DR remotely in a routine, mobile or tele-ophthalmology setting. Future studies could explore the use of more economical devices with smaller viewing resolutions to reduce cost implementation of DR screening services.
Keywords: diabetic retinopathy, screen resolution, screen size, screening
Introduction
Diabetic retinopathy screening programs have been implemented worldwide to enable early detection of diabetic retinopathy, which if treated appropriately, will minimize severe visual impairment.1 By 2030, it is estimated that the total number of people with diabetes will rise to 366 million.2 Because of the rising prevalence of diabetes, the current screening services in developing and developed countries will be faced with increasing costs of implementation and maintenance of a screening programme for the people with diabetes.3 It is therefore prudent that stakeholders continue to look for different ways of servicing the increasing diabetic population, and at the same time minimizing the economical impact of screening programs within the community.
To date, various studies have evaluated various parameters, which may affect the sensitivity, specificity, and cost effectiveness in screening diabetic retinopathy, including numbers of retinal fields,4 colour or monochromatic fundus photographs,5 mydriatic status,6, 7 photographers and readers with different medical qualifications,7 automated grading system,8 use of an economical retinal camera,9 and retinal video recording technique.10 However, none of the studies have compared the use of different viewing monitors for the reading and grading of diabetic retinopathy from the digital colour fundus photographs.
To date, the use of small viewing monitors in screening for diabetic retinopathy has become an emerging trend among the ophthalmologists or professional graders, as they can utilize them remotely without being physically present at the reading centre. Because of the growing popularity of these mobile and portable technologies, the purpose of our study is to evaluate the efficacy of using different portable and mobile devices with varying viewing monitors (15-inch MacBook Pro and 9.7-inch iPad) to detect subtle diabetic retinopathy changes (microaneurysms and dots haemorrhages) and diagnose the severity level. This also helps to determine the suitability of using smaller and more affordable portable devices to interpret the colour retinal images for grading of diabetic retinopathy.
Materials and methods
Design and data acquisition
This is a single-centre case series to evaluate different screening resolutions to interpret retinal colour still images for diabetic retinopathy screening. A total of 100 sets of non-stereo mydriatic three-field (optic disc, macula and temporal views) 35 degrees colour retinal still images consisting of 50 normal and 50 with diabetic retinopathy were selected into our study. The quality of all recruited retinal images were at least ‘acceptable' (more than two-third of retinal images were ‘interpretable') based on the reading outcomes using a 27-inch iMac (the standard viewing screen in our reading centre). All images were captured using FF 450 plus (Carl Zeiss, Inc., Oberkochen, Germany) by an experienced retinal photographer at the Diabetic Retinopathy Screening Clinic of Royal Perth Hospital and downloaded in Joint Photographic Experts Group (JPEG) format. The colour resolution of all images was fixed at 2588 × 1958 pixels. This study has been approved by the Royal Perth Hospital Human Research Ethics committee.
Data interpretation
All images were de-identified, randomized, and interpreted by two readers (a retinal specialist and a medical officer) in a dark room using the standardized Apple software—iPhoto (Apple, Cupertino, CA, USA) on three monitors with different sizes—27-inch iMac (Apple), 15-inch MacBook Pro (Apple), and 9.7-inch iPad (Apple). All images were interpreted on the specific three monitors with calibrated brightness at 100% target white point at D65; and target gamma at 2.2 using a software named Display Calibrator Assistant. The specification of the reading devices were listed in Table 1. The quality of retinal images was rated as ‘acceptable' or ‘uninterpretable' by the readers. The diabetic retinopathy severity level was graded based on presence/absence of microaneurysms, retinal haemorrhages, cotton wool spots, venous beading, intraretinal microvascular abnormalities, new vessels formation, and hard exudates using the International Clinical Diabetic Retinopathy Severity Scales11 (Table 2).
Table 1. Specifications and prices of 27-inch iMac, 15-inch MacBook Pro, and 9.7-inch iPad (the indicated prices are obtained in US dollars).
| 27-inch iMac | 15-inch MacBook Pro | 9.7-inch iPad (Wi-Fi+3G) | |
|---|---|---|---|
| Monitor screen (inch) | 27 | 15 | 9.7 |
| Screen resolution (pixels) | 1920 × 1080 | 1440 × 900 | 1024 × 768 |
| ATI Radeon HD | NVIDIA GeForce | ||
| Graphics processor | 4670 | 320M | A4 1Ghz |
| Weight (kg) | 13.8 | 2.54 | 0.73 |
| Price (USD) | $1699 | $1799 | $629 |
Table 2. International clinical diabetic retinopathy severity scales9.
| Grades | Retinal findings |
|---|---|
| None | No abnormalities |
| Mild NPDR | Microaneurysms only |
| Moderate NPDR | More than just microaneurysms |
| but less than severe NPDR | |
| Severe NPDR | Any of the following: |
| i. Extensive (>20) intraretinal haemorrhages in each of 4 quadrants | |
| ii. Definite venous beading in 2+ quadrants | |
| iii. Prominent IRMA in 1+ quadrant and no signs of proliferative DR | |
| Proliferative DR | One or more of the following: |
| i. Neovascularization | |
| ii. Vitreous/pretinal haemorrhage |
Abbreviations: NPDR, non-proliferative diabetic retinopathy; DR, diabetic retinopathy.
Sample size estimation
To allow for a power of 95%, desired precision of 0.10, expected sensitivity and specificity of 90%, the total number of eyes required for each device was 71 (prevalence was set at 0.50, as the selected samples consisted of 50% normal and 50% abnormal retinal colour still images).
Statistical analyses
All data were analysed using SPSS version 17 (SPSS, Chicago, IL, USA). The sensitivity, specificity, and Kappa correlation coefficient of 15-inch MacBook Pro and 9.7-inch iPad in detecting diabetic retinopathy lesions and grading were calculated with reference to the findings on the 27-inch iMac (as the reference standard). Moreover, the Kappa coefficient was performed on the diabetic retinopathy grading detected on 27-inch iMac for both readers and diabetic retinopathy lesions detected by 15-inch MacBook Pro and 9.7-inch iPad with reference to 27-inch iMac.
Results
The mean age (±SD) of the recruited participants was 51.3±13.8 years with HbA1c of 8.4±1.6% and duration of diabetes of 12.1±8.7 years. Of the retinal images, 50 (50%) had no diabetic retinopathy, 16 (16%) had mild non-proliferative diabetic retinopathy (NPDR), 25 (25%) had moderate NPDR, 7 (7%) had severe NPDR, and 2 (2%) had proliferative diabetic retinopathy. All retinal images were rated as ‘acceptable' by the retinal specialist and medical officer on 15-inch MacBook Pro and 9.7-inch iPad.
For 27-inch iMac (the ‘reference standard' of our study), both retinal specialist and medical officer had a Kappa correlation of 0.88 in detecting the overall diabetic retinopathy grading. In detection of any grade of diabetic retinopathy on 15-inch MacBook Pro, the retinal specialist had sensitivity and specificity of 96% and 96%, respectively, whereas the medical officer had 91.5% and 94.3%, respectively, with reference to the 27-inch iMac (Table 3). On the other hand, the sensitivity and specificity in detecting any grade of diabetic retinopathy on 9.7-inch iPad for retinal specialist were 91.8% and 94.1%, respectively, whereas for medical officer, they were 91.3 and 92.6% respectively. For sight-threatening diabetic retinopathy, the retinal specialist had 100% sensitivity and specificity on both reading devices, whereas for medical officer, the sensitivity, and specificity were 100% and 97.7%, respectively, on both devices.
Table 3. The sensitivity, specificity, and Kappa coefficient of 15-inch MacBook Pro and 9.7-inch iPad in detecting diabetic retinopathy grading by a retinal specialist and a medical officer with reference to 27-inch iMac.
| Any grade of diabetic retinopathy | Sensitivity (95% CI) | Specificity (95% CI) | Kappa |
|---|---|---|---|
| Retinal specialist | |||
| 15-inch MacBook Pro | 96% (85.1–99.3) | 96% (85.1–99.3) | 0.94 |
| 9.7-inch iPad | 91.8% (79.5–97.4) | 94.1% (82.8–98.5) | 0.89 |
| Medical officer | |||
| 15-inch MacBook Pro | 91.5% (78.7–97.2) | 94.3% (83.3–98.5) | 0.89 |
| 9.7-inch iPad | 91.3% (78.3–97.1) | 92.6% (81.3–97.6) | 0.88 |
| Sight-threatening diabetic retinopathy | |||
| Retinal specialist | |||
| 15-inch MacBook Pro | 100% | 100% | 1.00 |
| 9.7-inch iPad | 100% | 100% | 1.00 |
| Medical officer | |||
| 15-inch MacBook Pro | 100% | 97.7% | 0.92 |
| 9.7-inch iPad | 100% | 97.7% | 0.92 |
Sight-threatening diabetic retinopathy comprises of severe non-proliferative diabetic retinopathy and proliferative diabetic retinopathy.
The iPad had lower sensitivity and specificity (retinal specialist: 89.1% and 96.3%, respectively; medical officer: 87.5% and 98.1%, respectively) in detecting microaneurysms by both readers compared with MacBook Pro (sensitivity— retinal specialist: 100% and 96.3%, respectively; medical officer: 95.8% and 100%, respectively) (Table 4). Both devices had comparable sensitivity and specificity in detecting retinal haemorrhages by both readers (Table 4).
Table 4. The sensitivity, specificity and Kappa coefficient of 15-inch MacBook Pro and 9.7-inch iPad in detecting microaneurysms and retinal haemorrhages by a retinal specialist and a medical officer with reference to 27-inch iMac.
| Microaneurysms | Sensitivity (95% CI) | Specificity (95% CI) |
|---|---|---|
| Retinal specialist | ||
| 15-inch Macbook Pro | 100% (90.4–100) | 96.3% (86.1–99.4) |
| 9.7-inch iPad | 89.1% (75.6–95.9) | 96.3% (86.2–99.4) |
| Medical officer | ||
| 15-inch Macbook Pro | 95.8% (84.6–99.3) | 100% (91.4–100) |
| 9.7-inch iPad | 87.5% (74.1–94.8) | 98.1% (88.4–99.9) |
| Retinal haemorrhages | ||
| Retinal specialist | ||
| 15-inch Macbook Pro | 93.8% (77.8–98.9) | 98.5% (91–99.9) |
| 9.7-inch iPad | 93.8% (77.8–98.9) | 95.6% (86.8–98.9) |
| Medical officer | ||
| 15-inch Macbook Pro | 93.9% (78.4–98.9) | 98.5% (90.9–99.9) |
| 9.7-inch iPad | 93.9% (78.4–98.9) | 95.5% (86.6–98.8) |
For retinal specialist, the Kappa coefficient for 15-inch MacBook Pro and 9.7-inch iPad in detection of any grade of diabetic retinopathy were 0.94 and 0.89, respectively, whereas for medical officer, they were 0.89 and 0.88, respectively, with reference to 27-inch iMac. The Kappa coefficient in detecting all diabetic retinopathy signs (microaneurysms, retinal haemorrhages, cotton wool spots, new vessels formation, and hard exudates) by both readers were more than 0.80 (Table 5).
Table 5. The Kappa correlation of diabetic retinopathy changes interpreted by a retinal specialist and a medical officer on 15-inch MacBook Pro and 9.7-inch iPad with reference to the retinal findings detected on 27-inch iMac.
| Retinal findings |
Kappa statistics |
|
|---|---|---|
| 15-inch MacBook Pro | 9.7-inch iPad | |
| Retinal specialist | ||
| Microaneurysms | 0.96 | 0.86 |
| Retinal haemorrhages | 0.93 | 0.89 |
| Cotton wool spots | 1.00 | 1.00 |
| New vessels formation | 1.00 | 1.00 |
| Hard exudates | 0.96 | 0.96 |
| Cupped optic disc | 1.00 | 1.00 |
| Medical officer | ||
| Microaneurysms | 0.92 | 0.86 |
| Retinal haemorrhages | 0.93 | 0.91 |
| Cotton wool spots | 1.00 | 1.00 |
| New vessels formation | 1.00 | 1.00 |
| Hard exudates | 0.96 | 0.96 |
| Cupped optic disc | 1.00 | 1.00 |
Discussion
The success of a screening process relies on multiple factors, including the photographers' factor, patients' factor, and readers' factor. In the presence of an experienced photographer, patients with good ocular media, and experienced readers, the influence of viewing monitors also has a role in determining the diagnostic accuracy of retinal images grading. To evaluate the effectiveness of 15-inch Macbook Pro and 9.7-inch iPad in detecting diabetic retinopathy lesions and grading, we compared the retinal findings of each of the devices with the respective findings on 27-inch iMac. In our study, the retinal specialist and the medical officer as the trained reader had extremely strong inter-observer agreement (Kappa=0.88) in grading diabetic retinopathy on the 27-inch iMac. Both readers had excellent sensitivity and specificity in diagnosing diabetic retinopathy using 15-inch MacBook Pro (retinal specialist: 96%, 96%, respectively; medical officer: 91.5%, 94.3%, respectively) and 9.7-inch iPad (retinal specialist: 91.8%, 94.1%, respectively; medical officer: 91.3%, 92.6%, respectively) (Table 3). In addition, the Kappa correlation between 15-inch MacBook Pro and 9.7-inch iPad vs 27-inch iMac in detection of diabetic retinopathy changes (microaneurysms, retinal haemorrhages, cotton wool spots, neovascularization, and hard exudates) and diabetic retinopathy grading were excellent (greater than 0.8). These results indicated that the specialist (retinal specialist) and non-specialist (medical officer) screeners could effectively interpret and diagnose diabetic retinopathy from the colour retinal still images using a 15-inch or a 9.7-inch reading screen.
In detection of sight-threatening diabetic retinopathy (severe NPDR), the medical officer had 100% sensitivity and 97.7% specificity on both devices (MacBook Pro and iPad). The discrepancy of the specificity between the retinal specialist and the medical officer were due to two false positives, which had been graded by the medical officer as severe NPDR instead of moderate NPDR. Nevertheless, a screener especially the non-ophthalmologist personnel, such as the optometrists and general practitioners, should always be suspicious and have lower threshold in referring patients with uncertain diabetic retinopathy lesions detected on the retinal still images, even if this will result in some ‘unnecessary' referrals to the specialists.
The native image resolution of 2588 × 1958 pixels exceeded any of the compared displays spatial capabilities. This image resolution of 2588 × 1958 pixels was set by the fundus camera (Zeiss FF 450 plus) and all images were interpreted using a common software—iPhoto (Apple). The image size exceeded the compared displays spatial resolution and therefore, all the images were set to 100% to fit the full screen during the viewing and interpretation process. However, the full image may still be navigated on different screens using the iPhoto display programme with ease.
In our study, we utilized the 27-inch iMac as the reference standard of our study to avoid any diagnostic error secondary to small screen size and low screen resolution. We adopted the mydriatic 50 degrees three-field retinal still photography in our screening clinic given that its sensitivity in detecting any grade of diabetic retinopathy has been shown to be more than 90%. Compared with the gold standard seven-field 30 degrees stereoscopic views, it is more time saving and practical to be implemented in the routine screening setting. For the displaying programme, the ‘iPhoto' was utilized instead of the more specialized programs such as ‘Visupac' or ‘IMAGEnet system', as the latter programs often will need to be purchased. On the other hand, it is more economical to use ‘iPhoto' programme as it is a free software, which is included in the Mac computers.
In our study, the iPad had slightly lower sensitivity, specificity, and Kappa correlation with the 27-inch iMac compared with the 15-inch MacBook Pro (Table 4). Both devices, however, had excellent diagnostic accuracy in detecting sight-threatening diabetic retinopathy lesions (retinal haemorrhages, cotton wool spots, and new vessels) and diabetic retinopathy grading by both readers (Table 3). In a screening setting, it is rather critical to detect and refer patients with sight-threatening diabetic retinopathy changes, such as multiple retinal haemorrhages, cotton wool spots, and neovascularization, such that pan-retinal photocoagulation could be applied without delay to prevent severe visual impairment. Therefore, an occasional missed microaneurysm often will not result in severe visual impairment and this is consistent with the findings of our study given that both readers had 100% sensitivity in diagnosing sight-threatening diabetic retinopathy on both devices. Given the Kappa correlation between iPad and 27-inch iMac, graded by retinal specialist and medical officer, in detection of microaneurysms was within the excellent range (kappa=0.86), it will be feasible for the specialist and non-specialist readers to utilize a small reading screen (eg, 9.7-inch iPad) with a spatial resolution of 1024 × 768 pixels to effectively screen for diabetic retinopathy in the community. A further study will be of great value to explore the efficacy of other cheaper PC, tablet computers (eg, Galaxy Tab (Samsung, Samsung Town, Seoul, South Korea)) and cell/smart phones (eg, 3.5-inch iPhone (Apple) and 4-inch Galaxy S (Samsung)) with smaller reading screen sizes to screen for diabetic retinopathy in a routine, mobile, or tele-ophthalmology setting.
The strength of our study was being one of the recent studies that evaluated the effect of using devices with varying monitor resolution to screen for diabetic retinopathy. Moreover, we utilized two statistical methods (sensitivity/specificity and Kappa coefficient) and two readers (retinal specialist and medical officer) to justify the diagnostic accuracy of each monitor size by the specialist and non-specialist personnel. On the other hand, one of the weaknesses of our study was that the colour resolution of the three display monitors was different. Despite having a similar colour depth for all screens, the colour gamut, the range, and set of colours that they can produce, were not the same for all three monitors. The iMac can display much wider colour gamut than the other two displays used in this study. The Macbook Pro and iPad displays have much less display colour gamut than iMac. The colour gamut can influence the accuracy of the colours and may show undersaturated colours and hence, potentially affecting the interpretation of fundus images.
In addition, our results may be potentially subject to a selection bias, as we only selected the good quality colour retinal images into our study. It is unknown if the colour retinal images with suboptimal quality due to media opacity or dark fundi will affect the sensitivity and specificity in detecting diabetic retinopathy changes using the 15-inch or 9.7-inch reading screens. Thus, it will be of great significance to recruit all patients with diabetes consecutively from the screening clinic in the future study to evaluate the overall efficacy of different monitor resolutions in detecting diabetic retinopathy lesions from the colour retinal images with good and suboptimal quality.
Acknowledgments
Diabetes Australia Research Trust and Royal Perth Hospital have provided research funding to this project. The sponsor or funding organization had no role in the design or conduct of this research.
The authors declare no conflict of interest.
Footnotes
Author contributions
Daniel SW Ting contributed to the study conception and design, data acquisition, data analysis, data interpretation, and drafting the article. M L Tay-Kearney contributed to the study conception and design, revision of the important intellectual content, and final approval of the version to be published. J Vignarajan contributed to the data acquisition and data analysis. Y Kanagasingam contributed to the study conception and design, revision of the important intellectual content, and final approval of the version to be published.
References
- Ferris FL. How effective are treatments for diabetic retinopathy. JAMA. 1993;269:1290. [PubMed] [Google Scholar]
- Wild S, Roglic G, Green A, Sicree R, King H. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care. 2004;27:1047–1053. doi: 10.2337/diacare.27.5.1047. [DOI] [PubMed] [Google Scholar]
- James M, Turner DA, Broadbent DM, Vora J, Harding SP. Cost effectiveness analysis of screening for sight threatening diabetic eye disease. BMJ. 2000;320:1627–1631. doi: 10.1136/bmj.320.7250.1627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scanlon PH, Malhotra R, Greenwood RH, Aldington SJ, Foy C, Flatman M, et al. Comparison of two reference standards in validating two field mydriatic digital photography as a method of screening for diabetic retinopathy. Br J Ophthalmol. 2003;87:1258–1263. doi: 10.1136/bjo.87.10.1258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin D, Blumenkranz M, Brothers R, Grosvenor DM. The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic colour photography. Am J Ophthalmol. 2002;134:204–213. doi: 10.1016/s0002-9394(02)01522-2. [DOI] [PubMed] [Google Scholar]
- Hutchinson A, McIntosh A, Peters J, O'Keeffe C, Khunti K, Baker R, et al. Effectiveness of screening and monitoring tests for diabetic retinopathy—a systematic review. Diabet Med. 2000;17:495–506. doi: 10.1046/j.1464-5491.2000.00250.x. [DOI] [PubMed] [Google Scholar]
- Bragge P, Gruen RL, Chau M, Forbes A, Taylor HR. Screening for presence or absence of diabetic retinopathy: a meta-analysis. Arch Ophthalmol. 2011;129:435–444. doi: 10.1001/archophthalmol.2010.319. [DOI] [PubMed] [Google Scholar]
- Ting DSW, Tay-Kearney ML, Yogesan K. A light and portable novel device for diabetic retinopathy screening. Clin Experiment Ophthalmol. 2011;40:e40–e46. doi: 10.1111/j.1442-9071.2011.02732.x. [DOI] [PubMed] [Google Scholar]
- Ting DSW, Tay-Kearney ML, Constable IJ, Lim L, Preen DB, Kanagasingam Y. Retinal video recording: a new way to image and diagnose diabetic retinopathy. Ophthalmology. 2011;118:1588–1593. doi: 10.1016/j.ophtha.2011.04.009. [DOI] [PubMed] [Google Scholar]
- Wilkinson CP, Ferris FL, Klein RE, Lee PP, Agardh CD, Davis M, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology. 2003;110:1677–1682. doi: 10.1016/S0161-6420(03)00475-5. [DOI] [PubMed] [Google Scholar]
- Olson JA, Strachan FM, Hipwell JH, Goatman KA, McHardy KC, Forrester JV, et al. A comparative evaluation of digital imaging, retinal photography and optometrist examination in screening for diabetic retinopathy. Diabet Med. 2003;20:528–534. doi: 10.1046/j.1464-5491.2003.00969.x. [DOI] [PubMed] [Google Scholar]