Abstract
To validate optical coherence tomography (OCT) for the diagnosis of referable retinopathy (severe, very severe or proliferative retinopathy, and macular edema) in diabetic patients.
We performed a cross-sectional observational study. A random sample was analyzed comprising 136 eyes of diabetic patients referred to the hospital in Elche (Spain) with suspected referable retinopathy between October 2012 and June 2013. Primary variable: Referable retinopathy measured by ophthalmological examination of the retina. OCT data included: central foveal thickness, presence of intraretinal fluid, and fundus photographs. The receiver operating characteristic (ROC) curve was calculated to determine the minimum thickness value with a positive likelihood ratio >10. To determine the validity of OCT, the following diagnostic test was defined: Positive: if the patient had at least 1 of these criteria: foveal thickness greater than the point obtained on the previously defined ROC curve, intraretinal fluid, abnormal fundus photographs; Negative: none of the above criteria. Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and Kappa statistic were determined.
Of the 136 eyes, 48 had referable retinopathy (35.3%, 95% confidence interval [CI]: 27.3–43.3). The minimum thickness value with a positive likelihood ratio >10 was 275 μm. The diagnostic test constructed showed: sensitivity, 91.67% (95% CI: 79.13–97.30); specificity, 93.18% (95% CI: 85.19–97.20); positive predictive value, 88.00% (95% CI: 75.00–95.03); negative predictive value, 95.35% (95% CI: 87.87–98.50); positive likelihood ratio, 13.44 (95% CI: 6.18–29.24); negative likelihood ratio, 0.09 (95% CI: 0.03–0.23). The Kappa value was 0.84 (95% CI: 0.75–0.94, P < 0.001.
This study constructed a diagnostic test for referable diabetic retinopathy with type A evidence. Nevertheless, studies are needed to determine the validity of this test in the general diabetic population.
INTRODUCTION
Diabetic retinopathy is the leading cause of blindness in the population under 75 years of age in developed countries, and diabetic macular edema is the leading cause of vision loss.1,2
Recent decades have seen the development of strategies for diabetic retinopathy screening, such as the nonmydriatic fundus camera, though this has limitations for diagnosing diabetic macular edema.3–10 Retinal cameras capable of taking stereoscopic images have also been developed, but they require skilled technicians and a more costly infrastructure.11,12 A recently developed device, optical coherence tomography (OCT), provides quantitative evaluation rather than the qualitative assessment performed by macular biomicroscopy or fundus photography.13–16 OCT is also available with an incorporated retinal camera that enables both tests to be conducted with a single image capture.
Others have demonstrated the validity of OCT as a diagnostic method for macular edema in diabetic patients. However, these studies have not determined the validity of OCT to detect referable retinopathy (preproliferative, proliferative, and macular edema). We therefore conducted a study in a Spanish region assessing the validity of OCT, concurrently employing the quantitative value of OCT, fundus photography, and presence of retinal cysts. The aim of this study was to validate OCT as a method to accurately identify referable retinopathy.
METHODS
Study Population
The study population consisted of diabetic patients able to be seen at the outpatient ophthalmology department of the General University Hospital of Elche (Spain). This hospital provides free health coverage to a total of 169,555 inhabitants.
Design and Participants
This was an observational, cross-sectional study to determine the validity of a diagnostic test. A random sample was analyzed of patients diagnosed with diabetes mellitus (ICD-9-CM 250.X) referred to the general university hospital of Elche to be studied in compliance with the referral protocol (when the patient reports sudden loss of visual acuity, is suspected of having proliferative retinopathy, advanced retinopathy is shown on nonmydriatic retinography screening or there is evidence of cataracts) for diabetic patients in the period between October 2012 and June 2013. The sampling procedure consisted of determining a random day each week (not always the same day) and recruiting patients who came that day and chose to participate in the study, through linear systematic sampling. We excluded all patients who met any of the following criteria: dementia, cataract surgery in the last 3 months, laser treatment in the macular region or panphotocoagulation, antiangiogenic drugs, vitreoretinal surgery, high myopia, or other macular disorders.
Variables and Measures
The primary variable (gold standard) was referable retinopathy (preproliferative, proliferative, and macular edema) measured by clinical ophthalmological examination of the retina with indirect ophthalmoscopy and biomicroscopy of the central retina with a Topcon SL-8Z Slit Lamp (Topcon Corporation, Tokyo, Japan) through a 78 diopter lens (78D aspherical lens, Volk Optical Incorporated, Mentor, OH) and a 28D lens indirect ophthalmoscope. Macular edema was defined as the presence of hard exudates or retinal thickening located within a distance of 500 μm of the fovea, and the degree of diabetic retinopathy was defined according to the Early Treatment Diabetic Retinopathy Study (ETDRS) study classification.17 These assessments were performed by an ophthalmologist experienced in retinal disorders.
Three parameters were acquired from the OCT (Topcon 3D OCT-2000, Topcon Corporation®), which were interpreted by a different ophthalmologist and in a masked fashion with respect to the other screening tests. These parameters were: determination of central foveal thickness, the presence of intraretinal fluid, and fundus photographs.
To determine the foveal thickness, images were generated using 512 horizontal and 128 vertical scan lines comprised of 512 A-scans, applying the 6 × 6 mm 3D cube protocol centered on the fixation point after dilating the pupil with 10 mg/mL tropicamide. The mean retinal thickness was automatically calculated by the instrument software. For the measurement we used a 6 mm diameter area, the center of which coincided with the fovea, and this was used for evaluating the central 1000 μm area (the central circle).
The system performed a horizontal optical tomography image of the retina (B-Scan), which assessed the existence of intraretinal fluid (cysts or intraretinal spaces).
Fundus images taken by OCT included the macular region up to the nasal border of the optic nerve head at 45°, assessing diabetic retinopathy according to the classification established by the ETDRS. Preproliferative, proliferative, or nonassessable retinopathy was considered an abnormal finding. OCT takes a measurement of the 3 parameters at the same time, displaying results on a single screen (Figure 1).
FIGURE 1.
Screenshot of the retinal map analysis. The presence of intraretinal fluid is seen in the upper left of the image (B-scan). Both the fundus photograph and the central foveal thickness are shown in the upper right of the image. The copyright holder (Topcon Corporation) has approved the utilization of this figure.
Finally, at a descriptive level, the study recorded: age (years), diabetes evolution (years), HbA1c (%), fasting blood glucose (mmol/L), best corrected visual acuity, gender, type of diabetes mellitus, dyslipidemia, hypertension, and smoking status. These parameters were collected through the clinical history and patient interview, except corrected visual acuity, which was measured by the Snellen test.
Sample Size
The total sample size was 136 eyes, 48 of which had diabetic retinopathy or macular edema. To test for a Kappa statistic different from 0.4 and assuming a 95% confidence interval (CI), 35% agreement, and a Kappa coefficient of 0.8, the power of the test was 99.83%.18 The value of 0.4 was chosen because it represents moderate agreement.19
Statistical Methods
Qualitative variables were described by calculating absolute and relative frequencies, while quantitative variables were described by means and standard deviations.
The receiver operating characteristic (ROC) curve was calculated using macular edema as the status variable and foveal thickness as the continuous variable. The minimum thickness with a positive likelihood ratio strictly >10 was determined. This value was chosen because it provided type A evidence in the confirmation of a diagnostic test.20,21
To determine the validity of OCT the following diagnostic test was defined: Positive: if the patient met at least 1 of these criteria: foveal thickness greater than the point obtained from the ROC curve defined previously, intraretinal fluid, an abnormal fundus image; Negative: If none of the above criteria was met.
Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and the Kappa statistic were determined.
All analyses were conducted at a 5% significance level and for each relevant parameter its associated CI was calculated. The statistical package used was SPSS Statistics 19.0.
Ethical Issues
The study did not provide any additional risk to the patient. The study was conducted according to the principles of the World Medical Association Declaration of Helsinki and met the standards described in the European Union guidelines for good clinical practice. The patients were informed orally about the study and of the necessary information they had to provide. This study and the use of oral consent were approved by the Ethics Committee of the University General Hospital of Elche.
RESULTS
Of the initial 142 eyes of the diabetic patients included in our sample, 2 were excluded because they had epiretinal membrane, 2 due to vitreomacular traction syndrome, and 2 due to age-related macular degeneration. This left a final sample of 136 eyes, of which 48 had referable retinopathy (35.3%; 95% CI: 27.3–43.3) (12 with macular edema only, 6 with retinopathy only, and 30 with both conditions). Table 1 shows the descriptive information of the sample. Regarding OCT parameters, the mean foveal thickness was 268.6 μm, 36 eyes (26.5%) showed the presence of intraretinal fluid, and 28 had an abnormal fundus photograph (20.6%). With respect to the remaining descriptive features, we note that there was a higher proportion of type 2 diabetes (80.9%) and an elevated mean HbA1c (7.7%).
TABLE 1.
Descriptive Characteristics of the Diabetic Patients’ Eyes Analyzed: 2012 to 2013 Data
The area under the ROC curve (AUC) for central foveal thickness was 0.89 (95% CI: 0.81–0.97, P < 0.001) and its lowest value with a positive likelihood ratio >10 was 275 μm (Figure 2).
FIGURE 2.
ROC curve of central foveal thickness for diabetic retinopathy or macular edema. AUC, area under the ROC curve; CI, confidence interval.
The diagnostic test constructed was positive in 50 eyes (44 with referable retinopathy) and negative in 86 (82 without referable retinopathy). This produced the following parameters for the diagnostic test: sensitivity, 91.67% (95% CI: 79.13–97.30); specificity, 93.18% (95% CI: 85.19–97.20); positive predictive value, 88.00% (95% CI: 75.00–95.03); negative predictive value, 95.35% (95% CI: 87.87–98.50); positive likelihood ratio, 13.44% (95% CI: 6.18–29.24); and negative likelihood ratio, 0.09% (95% CI: 0.03–0.23). The kappa statistic was 0.84% (95% CI: 0.75–0.94, P < 0.001).
DISCUSSION
Our study evaluated the validity of OCT as a screening method while also assessing the 3 parameters it determined: central foveal thickness, fundus photography, and the presence of intraretinal fluid. In this validation, very relevant clinical parameters were obtained, since the likelihood ratios had type A evidence, both to confirm and to rule out the diagnosis of referable retinopathy.20,21 Furthermore, once the role of chance (Kappa index) was removed, correlation with the gold standard method was 84.1%, which supports the good clinical indicators found.
It was difficult to compare our results with those of others because we found no studies analyzing the validity of OCT while at the same time taking into account the 3 parameters used in our paper and detecting both retinopathy and macular edema in the diabetic patient. In the meta-analysis by Virgili et al, the validity of the foveal thickness measured with OCT was evaluated to detect diabetic macular edema and a pooled sensitivity of 78% and a specificity of 86% (likelihood ratio: positive, 5.57; negative, 0.26) were determined.22 In another meta-analysis, by Bragge et al, the pooled sensitivity and specificity of diagnostic tests were calculated to detect diabetic retinopathy with mydriasis.23 Sensitivity was 84.5% and specificity was 88.6% (likelihood ratio: positive, 7.41; negative, 0.17).
The results of this paper show that by using all the components of OCT, a diagnostic test with type A evidence is obtained that can be used to diagnose referable retinopathy. If a patient is referred to the ophthalmology department with suspected referable retinopathy, it would be advisable to perform the diagnostic test constructed herein. In the case of a negative result, the patient would be referred to their primary care physician to be followed as per protocol. On the other hand, if the test is positive, the patient would be reviewed and treated at the ophthalmology department.
Strengths and Limitations of the Study
The main strength of this study is that it innovatively constructed a diagnostic test with type A evidence able to diagnose referable retinopathy in patients who had already been referred to the ophthalmology department for suspected diabetic retinopathy. Moreover, once the role of chance was eliminated, the correlation between the gold standard and our diagnostic test was 0.84, representing excellent agreement.24 In addition, the statistical power to compare the value of this index with 0.4 (moderate agreement) was close to 100%.
Concerning limitations, we note that the patients analyzed already had suspected diabetic retinopathy; therefore, studies to determine the validity of OCT, using its 3 components, in the general diabetic population would be of interest. To minimize selection bias, a random sample of patients was selected from the study population analyzed. Finally, regarding information bias, calibrated instruments were used by expert ophthalmologists.
CONCLUSION
This study contributes a diagnostic test for referable diabetic retinopathy based on the 3 components of OCT, with type A evidence, to confirm or rule out the disease. However, these results have been obtained in a referred population. Consequently, studies are needed to determine the validity of this test in the general diabetic population.
ACKNOWLEDGMENT
The authors thank Maria Repice and Ian Johnstone for their help with the English language version of the text.
Footnotes
Abbreviations: AUC = area under the ROC curve, CI = confidence interval, ETDRS = Early Treatment Diabetic Retinopathy Study, OCT = optical coherence tomography, ROC = receiver operating characteristic.
The authors have no funding and conflicts of interest to disclose.
REFERENCES
- 1.Aiello LP, Gardner TW, King GL, et al. Diabetic retinopathy. Diabetes Care 1998; 21:143–156. [DOI] [PubMed] [Google Scholar]
- 2.Klein R, Moss SE, Klein BE, et al. The Wisconsin epidemiologic study of diabetic retinopathy. XI. The incidence of macular edema. Ophthalmology 1989; 96:1501–1510. [DOI] [PubMed] [Google Scholar]
- 3.Kohner EM, Porta M. Protocols for screening and treatment of diabetic retinopathy in Europe. Eur J Ophthalmol 1991; 1:45–54. [DOI] [PubMed] [Google Scholar]
- 4.Chew EY, Benson WE, Boldt HC, et al. Diabetic Retinopathy. Preferred Practice Pattern. San Francisco: American Academy of Ophthalmology; 2003. [Google Scholar]
- 5.Gibbins RL, Owens DR, Allen JC, et al. Practical application of the European Field Guide in screening for diabetic retinopathy by using ophthalmoscopy and 35 mm retinal slides. Diabetologia 1998; 41:59–64. [DOI] [PubMed] [Google Scholar]
- 6.Aldington SJ, Kohner EM, Meuer S, et al. Methodology for retinal photography and assessment of diabetic retinopathy: the EURODIAB IDDM complications study. Diabetologia 1995; 38:437–444. [DOI] [PubMed] [Google Scholar]
- 7.Gomez-Ulla F, Fernández MI, González F, et al. Digital retinal images and teleophthalmology for detecting and grading diabetic retinopathy. Diabetes Care 2002; 25:1384–1389. [DOI] [PubMed] [Google Scholar]
- 8.Baeza M, Orozco-Beltrán D, Gil-Guillen VF, et al. Screening for sight threatening diabetic retinopathy using non-mydriatic retinal camera in a primary care setting: to dilate or not to dilate? Int J Clin Pract 2009; 63:433–438. [DOI] [PubMed] [Google Scholar]
- 9.Herbert HM, Jordan K, Flanagan DW. Is screening with digital imaging using one retinal view adequate? Eye 2003; 17:497–500. [DOI] [PubMed] [Google Scholar]
- 10.Baeza Diaz M, Gil Guillen V, Orozco Beltran D, et al. Validez de la cámara no midriática en el cribado de la retinopatía diabética y análisis de indicadores de riesgo de retinopatía. Arch Soc Esp Oftalmol 2004; 79:433–442. [DOI] [PubMed] [Google Scholar]
- 11.Bursell SE, Cavallerano JD, Cavallerano AA, et al. Stereo nonmydriatic digital-video color retinal imaging compared with Early Treatment Diabetic Retinopathy Study seven standard field 35-mm stereo color photos for determining level of diabetic retinopathy. Ophthalmology 2001; 108:572–585. [DOI] [PubMed] [Google Scholar]
- 12.Rudinsky CJ, Hinz BJ, Tennant MT, et al. High-resolution stereoscopic digital fundus photography versus contact lens biomicroscopy for the detection of clinically significant macular edema. Ophthalmology 2002; 109:267–274. [DOI] [PubMed] [Google Scholar]
- 13.Browning DJ, McOwen MD, Bowen RM, Jr, et al. Comparison of the clinical diagnosis of diabetic macular edema with diagnosis by optical coherence tomography. Ophthalmology 2004; 111:712–715. [DOI] [PubMed] [Google Scholar]
- 14.Koozekanani D, Roberts C, Katz SE, et al. Intersession repeatability of macular thickness measurements with the Humphrey 2000 OCT. Invest Ophthalmol Vis Sci 2000; 41:1486–1491. [PubMed] [Google Scholar]
- 15.Hee MR, Puliafito CA, Wong C, et al. Quantitative assessment of macular edema with optical coherence tomography. Arch Ophthalmol 1995; 113:1019–1029. [DOI] [PubMed] [Google Scholar]
- 16.Hee MR, Puliafito CA, Duker JS, et al. Topography of diabetic macular edema with optical coherence tomography. Ophthalmology 1998; 105:360–370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Early Treatment Diabetic Retinopathy Research Group. Grading diabetic retinopathy from stereoscopic color fundus photographs: an extension of the modified Airlie House classification. ETDRS report number 10. Ophthalmology 1991; 98: Suppl:786–806. [PubMed] [Google Scholar]
- 18.Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Stat Med 1992; 11:1511–1519. [DOI] [PubMed] [Google Scholar]
- 19.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159–174. [PubMed] [Google Scholar]
- 20.Guyatt G, Drummond R. Users’ Guides to the Medical Literature: A Manual for Evidence-based Clinical Practice. Chicago, IL: AMA Press; 2002. [Google Scholar]
- 21.Ramírez-Prado D, Palazón-Bru A, Folgado-de-la Rosa DM, et al. Predictive models for all-cause and cardiovascular mortality in type 2 diabetic inpatients. A cohort study. Int J Clin Pract 2015; 69:474–484. [DOI] [PubMed] [Google Scholar]
- 22.Virgili G, Menchini F, Murro V, et al. Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev 2011; 7:CD008081. [DOI] [PubMed] [Google Scholar]
- 23.Bragge P, Gruen RL, Chau M, et al. Screening for presence or absence of diabetic retinopathy: a meta-analysis. Arch Ophthalmol 2011; 129:435–444. [DOI] [PubMed] [Google Scholar]
- 24.Fleiss JL. Statistical Methods for Rates and Proportions. New York: John Wiley; 1981. [Google Scholar]