Imager Evaluation of Diabetic Retinopathy at the Time of Imaging in a Telemedicine Program

Abstract

OBJECTIVE

To evaluate the ability of certified retinal imagers to identify presence versus absence of sight-threatening diabetic retinopathy (stDR) (moderate nonproliferative diabetic retinopathy or worse or diabetic macular edema) at the time of retinal imaging in a telemedicine program.

RESEARCH DESIGN AND METHODS

Diabetic patients in a primary care setting or specialty diabetes clinic received Joslin Vision Network protocol retinal imaging as part of their care. Trained nonphysician imagers graded the presence versus absence of stDR at the time of imaging. These gradings were compared with masked gradings of certified readers.

RESULTS

Of 158 patients (316 eyes) imaged, all cases of stDR (42 eyes [13%]) were identified by the imagers at the time of imaging. Six eyes with mild nonproliferative diabetic retinopathy were graded by the imagers to have stDR (sensitivity 1.00, 95% CI 0.90–1.00; specificity 0.97, 0.94–0.99).

CONCLUSIONS

Appropriately trained imagers can accurately identify stDR at the time of imaging.

The American Telemedicine Association Telehealth Practice Recommendations for Diabetic Retinopathy identifies four categories of telemedicine care for diabetic retinopathy (1). Category 1 programs identify patients with no or minimal diabetic retinopathy (Early Treatment Diabetic Retinopathy Study [ETDRS] level 20 or below) versus those with diabetic retinopathy more severe than ETDRS level 20. Category 2 programs accurately determine whether sight-threatening diabetic retinopathy (stDR), as evidenced by any level of diabetic macular edema (DME), severe or worse levels of nonproliferative diabetic retinopathy (NPDR) (ETDRS level 53 or worse), or proliferative diabetic retinopathy (ETDRS level 61 or worse), is present or not present. Category 3 programs accurately identify ETDRS-defined levels of diabetic retinopathy and DME to determine appropriate follow-up and treatment. Category 4 programs can replace ETDRS 7-standard field 35-mm stereoscopic color fundus photographs in any clinical or research program.

The Joslin Vision Network (JVN) is a validated category 3 program (2–5). Imagers undergo an intensive 3-day program that includes fundus camera operation and imaging software navigation; structured courses on diabetes, ocular anatomy, diabetic retinopathy, and common ocular disorders; and a guided review demonstrating retinal images of nondiseased and diseased eyes. As part of the certification, imagers learn to recognize lesions of diabetic retinopathy, including hemorrhages, microaneurysms, venous caliber abnormalities, intraretinal microvascular abnormalities, retinal neovascularization, cotton wool spots, hard exudates, and laser scars. Salient retinal abnormalities not related to diabetes are also demonstrated, including choroidal nevi, retinal emboli, and large or asymmetrical optic cup-to-disc ratios. After the 3-day program, imagers serve a probationary period with senior imager supervision and ongoing quality improvement and assurance.

This prospective study assessed the ability of two certified imagers to conduct American Telemedicine Association Category 2 (presence vs. absence of stDR) grading at the time of retinal imaging.

RESEARCH DESIGN AND METHODS

Patients with diagnosed diabetes had nonmydriatic JVN imaging as part of their routine physical examinations in a primary care setting (HealthCare Associates, Beth-Israel Deaconess Medical Center) or a specialty diabetes clinic (Adult Diabetes, Joslin Diabetes Center). At the time of imaging, certified imagers (A.M.T., 71 patients [45%]; T.F., 87 patients [55%]) identified patients with potential stDR, defined for this program as ETDRS levels of 43 or worse (6) or DME, and ungradable images. Imagers were not able to manipulate the color, brightness, contrast, or other features of the images and could not view images stereoscopically. To grade retinal thickening without stereoscopic viewing, imagers relied on identifying hard exudates or microaneurysms within 3,000 microns from the center of the macula as surrogate markers for DME. The two certified imagers were Bachelor of Arts college graduates with no prior health care experience in evaluating retinal images and had not provided direct patient care before working as retinal imagers. Certified readers graded images according to the previously described JVN protocol (2,3) in a central reading center with calibrated monitors and stereoscopic viewing capability. All readers in the JVN program are Massachusetts-licensed optometrists. All readers were masked to the grading performed by the imagers. All findings were recorded on a specifically designed template.

RESULTS

A total of 158 consecutive patients were imaged. Mean age was 56.5 years (range 22–86), 54% female, and the mean diabetes duration was 7.0 years (range 0.1–42). A total of 316 eyes were evaluated, and 195 (61.7%) had no diabetic retinopathy, 62 (19.6%) had mild NPDR, 24 (7.6%) had moderate NPDR, 3 (1%) had severe or very severe NPDR, 2 (0.6%) had proliferative diabetic retinopathy, and 30 (9.5%) were ungradable for diabetic retinopathy. DME was absent in 266 (84.2%) eyes, present in 13 (4.1%), and 37 (11.7%) were ungradable for DME.

Of the 316 eyes assessed, imagers identified 48 (15%) eyes with potential stDR at the time of imaging. Subsequent grading by certified readers classified 6 (12.5%) of these eyes as mild NPDR. The imagers accurately identified all cases of stDR as graded by the readers. Although limited by the moderate sample size and the use of only two independent imagers, the agreement for determining stDR between imagers and readers was 0.95 ± 0.02. The sensitivity and specificity in identifying stDR at the time of imaging by a certified imager is 1.00 (95% CI 0.90–1.00) and 0.97 (95% CI 0.94–0.99), respectively (positive predictive value 0.88 [95% CI 0.74–0.95]; negative predictive value 1.00 [0.98–1.00]). There was complete agreement between imagers and readers regarding ungradable eyes (37 [12%]). Table 1 presents a cross-tabulation of imager and reader evaluations for the presence of stDR and ungradable images.

Table 1.

Cross-tabulation of grading for the presence of stDR by JVN imagers and JVN readers

CONCLUSIONS

Film or digital retinal imaging is a sensitive method to identify the presence and level of diabetic retinopathy (7–10). Despite efforts to automate retinal image evaluation (11–13), currently no system can perform such analyses in real time, and present methods of retinal imaging require trained imagers to acquire retinal images.

This study shows that appropriately educated and certified imagers following a clearly defined imaging and grading protocol can accurately evaluate retinal images with a high degree of sensitivity and specificity for the presence of stDR and inadequate image quality at the time of imaging. The ability to identify ungradable images and detect potential stDR facilitates reacquisition of retinal images during a single imaging encounter and allows prompt referral to appropriate eye care. Although this study involved a moderate number of eyes (n = 316), 42 (13%) eyes with stDR and 37 (12%) eyes with ungradable images were identified, representing all cases that would have required further ophthalmic evaluation and care. Additional studies with a variety of imagers and patient populations will be required to determine whether similar results can be obtained across diverse health care scenarios. However, the fact that the two certified imagers involved in this study had no prior health care experience in evaluating retinal images suggests that similar results are possible. In this study, retinal imagers had received a validated standardized method of certification and training, which is an important consideration when extrapolating these results to other retinal imaging programs.