Enhancing Risk Assessment in Patients with Diabetic Retinopathy by Combining Measures of Retinal Function and Structure

Abstract

Purpose

To determine whether combining measures of retinal structure and function predicts need for intervention for diabetic retinopathy (DR) better than either modality alone.

Methods

The study sample consisted of 279 diabetic patients who participated in an earlier cross-sectional study. Patients were excluded if they were previously treated for macular edema or proliferative DR or if they had other retinopathies. Medical records were reviewed for ocular interventions including vitrectomy, intravitreal injection, and laser treatment. Need for intervention was analyzed using Kaplan-Meier analyses and Cox proportional hazards. Baseline electroretinograms and fundus photographs were obtained. Two definitions of structural positive findings were as follows: 1. Early Treatment of Diabetic Retinopathy Study diabetic retinopathy severity scale (ETDRS-DR) severity ≥ level 53 (ETDRS-DR+) and 2. ETDRS-DR+ or clinically significant macular edema (VTDR+). A positive function finding corresponded to a RETeval DR Score >23.5 (RETeval+).

Results

For patients with VTDR+ the incidence of intervention was 19%, 31%, and 53% after 1, 2, and 3 years of follow-up. In these patients, intervention incidence increased to 34%, 54%, and 74% the subsequent 1, 2, and 3 years if function was above criterion (RETeval+), whereas RETeval−  results reduced the risk to 3%, 4%, and 29%, respectively, reducing risk to similar levels seen for patients with VTDR− results at baseline.

Conclusions

Prediction of subsequent intervention was best when combining structural and functional information.

Translational Relevance

This study demonstrates that clinical management of diabetic retinopathy is improved by adding electroretinography to fundus photographic information in assessing the risk of the need for intervention.

Introduction

Diabetic retinopathy (DR) is the leading cause of blindness in working age Americans and is a major cause of blindness worldwide.^1,2 Recent studies have shown that diabetic eye disease is underdiagnosed with many barriers to appropriate testing and treatment referrals.^3–5 Increasing testing efficiency (i.e., the percentage of people with diabetes who are tested for eye disease) reduces concomitant vision loss.⁶

Photographic systems for DR detection use readers (human or artificial intelligence^7,8) to review fundus photographs to identify structural features of DR and predict risk for visual loss based on the severity of these features. Although it has long been recognized that characteristics of fundus imaging predict the risk of subsequent visual loss in diabetic patients, this relationship is not perfect. Some eyes with mild DR will progress to clinically significant macular edema (CSME) or proliferative diabetic retinopathy (PDR) within a year, whereas ∼50% of eyes with severe nonproliferative diabetic retinopathy (NPDR) will not progress to CSME/PDR over a year.^9,10 Even patients with high-risk PDR have only a 15.8% (95% confidence interval [CI]: 13.6%–18.4%) chance of having severe vision loss or vitrectomy within five years after observation of high-risk PDR.⁹

An alternative method to detect diabetic eye disease is with an electroretinogram (ERG). The ERG is a measure of functional integrity of the retina and is sensitive to retinal ischemia.^11,12 Previous studies have shown that ERG oscillatory potentials, 30 Hz flicker implicit time, and implicit time of the b-wave of the photopic ERG have predictive value in diabetic eyes.^13,14 A DR score derived from flicker ERG and pupillography data obtained from a handheld RETeval ERG device has also been shown to correlate with funduscopic DR severity.^15–18

In this study, longitudinal outcome data were analyzed in a sample of patients with DR who were previously tested with functional (ERG, pupillary light reflex) and structural (stereo fundus photographs) measurements on the same day in a cross-sectional study.¹⁵ The outcomes of interest were medical or surgical ocular interventions required to treat DR including intravitreal injections (IVT), laser treatment, and vitrectomy. Predictive performance of the functional measures, structural measures, and their combination were assessed. We hypothesize that the combination of functional and anatomic data will be more predictive of the necessity for DR intervention than either modality alone.

Methods

A retrospective chart review was performed on the 279 diabetic patients at the Atlanta VA Medical Center initially recruited between 2013 and 2014 for a cross-sectional study¹⁵ comparing ERG results from the RETeval device with DR severity from stereo fundus photographs (Study NCT01950663 ClinicalTrials.gov). The Atlanta VA Medical Center and Emory School of Medicine Institutional Review Board approved the initial study and the subsequent amendment to perform this chart review of all patients previously enrolled in the initial clinical trial at this site. Research was conducted according to the tenets of the Declaration of Helsinki and the study conformed to the Health Information Privacy and Portability Act (HIPPA). Written informed consent was obtained from all participants.

The cross-sectional study¹⁵ was used to derive and validate a functional DR Score algorithm derived from flicker ERG and pupillary light reflex data. Patients were selectively recruited to have a wide range of disease severity, and patients previously treated for macular edema or proliferative DR and those with other retinopathies were excluded. Light-adapted flicker electroretinogram and pupillary light reflex data were obtained without dilation using the RETeval portable system (LKC Technologies, Gaithersburg, MD, USA). Responses were obtained to 28.3 Hz flicker flash stimuli of strength 4, 8, 16, and 32 Troland seconds (Td·s). Mean testing time was 2.3 minutes to test both eyes. ERG timing and amplitude, as well as steady-state pupillary constriction, were measured at each flicker strength. Results were combined to form the RETeval DR Score optimized for the detection of vision-threatening DR (vision-threatening DR or VTDR defined as severe NPDR, PDR, or CSME). The DR Score combines the larger ERG amplitude of the two eyes, the shorter ERG flicker implicit time between the two eyes, the smaller ratio of steady-state pupil area to a dim (4 Td·s) and bright (32 Td·s) flickering light, and the patient's age to generate a dimensionless, continuous value that increases with increasing DR.¹⁵ The DR Score is automatically generated by the RETeval device's DR Assessment protocol. Seven-field stereo color photos were obtained after pupil dilation. The photos were double graded using the Early Treatment of Diabetic Retinopathy Study diabetic retinopathy severity scale (ETDRS-DR severity) in a dedicated reading center, with adjudication for results that differed by more than one level or differed on the assessment of VTDR.

Patient charts were reviewed up to four years after the initial examination. Ophthalmology and optometry clinic notes filed in the VA Computerized Patient Record System were analyzed. Interventions for progression of DR or diabetic macular edema (DME) were documented (IVT, vitrectomy, pan retinal photocoagulation [PRP], focal or grid laser). If an intervention was needed, the date that the intervention was approved or given was used, whichever came first. The individual performing the chart review (B.C.) was masked to original measurements. The mean follow-up time was 2.4 ± 0.9 (0.1–4.2) years (mean ± standard deviation, [range]). The number of patients followed up for at least one, two, and three years were 231, 186, and 95, respectively.

Prognostic Criteria

Two structural prognostic criteria were used:

• 1.All patients with severe NPDR or PDR; i.e., ETDRS-DR severity ≥ 53 were classified as ETDRS-DR+
• 2.All patients with ETDRS-DR+ and all patients with clinically significant macular edema were classified as VTDR+

The VTDR+ classification matches the AAO preferred practice criterion for immediate referral for diabetic retinopathy.¹⁹ The ETDRS-DR classification was also evaluated since typical screening regimens do not use the stereo photography required to measure CSME.

The retinal function criterion used in this study was a RETeval DR Score ≥ 23.5 (RETeval+). This measure was derived by Maa et al.¹⁵ to optimize ERG and pupillography results from the RETeval device to aid in the detection of vision-threatening diabetic retinopathy. A cutoff of 23.5 optimized performance for this longitudinal dataset.

All criteria also include ungradable results, as it is safer to consider ungradable results in the “+” category.

Statistical Methods

Patients were classified as positive or negative for each criterion. Kaplan-Meier²⁰ analysis was performed to compare the rates of ocular interventions between groups. A log rank test with Tarone-Ware weighting was used for significance testing of the Kaplan-Meier curves, where the Tarone-Ware weighting was selected to provide more weighting for earlier time points. To quantify relative risk (RR), Cox proportional hazards models were created after right-censoring at two years, which prevents test performance past two years from affecting RRs, because it is unlikely that clinical follow-up intervals would be greater than this time period. For RETeval results, RRs were computed on measurements as continuous variables and at the 23.5 DR score threshold, whereas the photographic results were computed at thresholds only. The exact marginal likelihood method was used to handle ties. Models that combine a RETeval result and a photographic result were also computed. Incidence standard errors of the mean and p-values were based on Bernoulli distributions. P-values less than 0.05 were considered significant. Statistical analysis was performed using Mathematica version 12.1 (Wolfram Research, Champaign, IL, USA).

Results

No follow-up data was available for 27 patients, leaving 252 patients in the sample analyzed in this study. Baseline patient demographics are summarized in Table 1. Patient age was 61 years (standard deviation 9, range 35–88). After one, two, and three years of follow-up, 7%, 11%, and 17% of patients had an intervention to treat progression of DR (vitrectomy, PRP, focal or grid laser, IVT), respectively (Fig. 1A). The most common intervention was laser (occurring in 4.5%, 7.4%, and 12% of patients at years 1, 2, and 3, respectively), followed by IVT (occurring in 2.5%, 4.9%, and 8.3% of patients), and vitrectomy (0.9%, 1.45, and 5% of patients). The aggregated incidence of ocular interventions was used for all analyses.

Table 1.

Patient Characteristics

Parameter	Number of Subjects	%
Race
African American	137	54
Caucasian	95	38
Native American	11	4
No answer	6	2
Other	2	1
Asian	1	0.4
Ethnicity
Hispanic	8	3
Sex
Male	213	85
Female	39	15
Diabetes medication
Oral	71	28
Injection	92	37
Both	89	35
Edema
None	167	66
CSME	50	20
Macular edema	21	8
Ungradable	14	6
International clinical classification2 (ETDRS level)
No DR (10)	38	15
Mild NPDR (14)	1	0.4
Mild NPDR (15)	11	4
Mild NPDR (20)	9	4
Mild NPDR (35)	47	19
Moderate NPDR (43)	41	16
Moderate NPDR (47)	35	14
Severe NPDR (53)	8	3
Proliferative DR (61)	13	5
Proliferative DR (65)	8	3
Proliferative DR (71)	2	1
Proliferative DR (85)	1	0.4
Ungradable	38	15

Figure 1.

(A) Percentage of patients without intervention as a function of time. Shaded area is the 95% confidence interval. (B) Percentage of patients without intervention as a function of RETeval DR Score (ERG/pupillography) at years 1, 2, and 3. The vertical grid line shows the cutoff point of 23.5. (C) Percentage of patients without intervention as a function of ETDRS-DR severity. The vertical grid line shows the cutoff point of ETDRS-DR severity 53 (severe NPDR). Also shown are results from the 15% of patients with ungradable photographs. The single patient with ungradable RETeval DR Score did not have an intervention and is not shown.

Both baseline ETDRS-DR severity (from fundus photographs) and baseline DR score (ERG and pupillary light reflex) were associated with the likelihood of progression at one, two, and three years of follow-up (Figs. 1B, 1C). Kaplan-Meier survival curves illustrate the predictive value of structural and functional measures (Fig. 2). All structural and functional parameters have statistically significant value in predicting which patients will have an ocular intervention (P < 0.0001). Compared with the a priori results (i.e., ignoring the baseline measurements), a negative result reduced the risk, and a positive result increased the risk of an ocular intervention.

Figure 2.

Percentage of patients without an ocular intervention as a function of time. The red curves in each panel show patients that are positive for the criterion given in the title of each panel, whereas the green curves represent patients who were classified as negative for the criterion. The black curve is the a priori curve from Figure 1A. The P values represent the likelihood the performance difference between the two classification groups is chance. Shaded areas are the 95% CI.

A Cox proportional hazards analysis for the RETeval DR Score shows that for each unit increase in the RETeval DR Score, the risk of ocular intervention increases by 1.28 × (95% CI: 1.17–1.40, P < 0.0001). This risk is multiplicative. For example, comparing a patient with a DR score 26 to a patient with a DR score of 16 (a difference of 10), the former has a 1.28¹⁰ = 12× greater risk of having an ocular intervention. The DR Score increment needed to increase the risk by 2×, 5×, and 10× is 2.8, 6.5, and 9.3, respectively. A similar analysis cannot be done with the anatomic parameters, because they are not continuous variables.

Patients can be classified as positive or negative for each criterion and the RR associated with being positive or negative can be ascertained (Fig. 3). Eyes with a RETeval DR Score ≥ 23.5 (RETeval+) have an 11 times greater risk of having an ocular intervention compared with those with a DR Score < 23.5 (RETeval−), whereas eyes with ETDRS-DR severity ≥ 53 (ETDRS-DR+) have a 3.5 times greater risk compared with those with moderate NPDR or milder DR (ETDRS-DR−). Although the risks for all criteria are significantly different from 1 (p ≤ 0.002), they are not statistically different from one another, We also examined ETDRS-DR severity thresholds of ≥  47 and ≥  61 and found no significant difference from the ETDRS-DR severity ≥ 53 threshold (RRs of 4.8 and 4.2, respectively). Adding CSME is an important addition to the RR (Table 2). Patients with moderate NPDR or milder DR with CSME have a trend toward higher risk (3.1×, P = 0.07) of having an ocular intervention than those without CSME, while patients with CSME and either severe NPDR or PDR (VTDR+) have a 4.7× higher risk (P = 0.0008).

Figure 3.

Relative risk for binary classification of patients for having an ocular intervention. Error bars are the 95% confidence interval. The P values represent the likelihood that the relative risk is 1.

Table 2.

Relative Risk of Ocular Intervention

		ETDRS-DR
		Negative	Positive
	Positive	RR = $5_{-4}^{+12}$	RR = ${15}_{-9}^{+24}$
RETeval		P = 0.005	P < 0.00001
		n = 27	n = 30
	Negative	RR = 1	RR = $0.6_{-0.5}^{+5}$
		n = 155	P = 0.7
			n = 40
		VTDR
		Negative	Positive
	Positive	RR = $1.5_{-1.3}^{+11}$	RR = ${12}_{-7}^{+19}$
		P = 0.7	P = 0.00001
RETeval		n = 15	n = 42
	Negative	RR = 1	RR = $0.{38}_{-0.33}^{+2.8}$
		n = 136	P = 0.4
			n = 59
		VTDR
		Negative	Positive
	Positive	N/A	RR = $4.7_{-2.8}^{+7}$
		n = 0	P = 0.0008
ETDRS-DR			n = 70
	Negative	RR = 1	RR = $3.1_{-2.2}^{+8}$
		n = 151	P = 0.07
			n = 31

Values to the right of the RR indicate the values to add or subtract to obtain 95% confidence interval. RR, relative risk; n, number of subjects.

Combining structural and functional measurements significantly improved predictive value compared to the individual parameters (Table 2 and Fig. 4). Functional measurements modulate the risk of future ocular interventions with both negative and positive structural results. Patients who are RETeval+ and ETDRS-DR− have a fivefold greater risk of intervention than patients who are negative for both tests (P = 0.005), and patients who are positive on both tests have a 15-fold greater risk (P < 0.00001). On the other hand, patients with ETDRS-DR+/RETeval− have similar risk to patients with ETDRS-DR- (P = 0.7). This is also observable in the left Kaplan-Meier plots of Figure 4, where the functional test negative curves (in green) overlap, suggesting independence from the ETDRS-DR severity results. When adding CSME to the ETDRS levels as the structural criterion, a negative result from either structure or function puts patients at same risk as patients that had negative results on both tests (i.e., one negative result overrules a positive result from the other test). Both tests being positive, on the other hand, puts patients at a RR of 12 (P < 0.00001).

Figure 4.

The effect of the combination of structural and functional measures on the percentage of patients without an ocular intervention as a function of time. Both panels use the functional RETeval DR Score while the structural measurement is either ETDRS-DR severity (left panel) or presence of VTDR (right panel). Positive/negative structural measurements are solid/dashed lines, respectively, whereas positive/negative functional measurements are red/green, respectively. The black curve is the a priori curve from Figure 1A.

An examination of the proportion of positive cases illustrates the prognostic value of adding functional measurements to structural measurements. As illustrated in Figure 5B, 9/66 (14%) of patients with ETDRS-DR+ had an ocular intervention within one year, 14/52 (27%) and 20/31 (65%) had an ocular intervention after two and three years (some patients were lost to follow-up between the two- and three-year time points). Of these patients, if they were RETeval DR+ the proportion of patients needing intervention increased to 8/27 (30%) at one year, 13/22 (59%) at two years, and 15/18 (83%) at three years. Alternatively, if they were RETeval DR− their chance of having an intervention was reduced to 1/39 (3%) at one year, 1/30 (3%) at two years, and 5/13 (38%) at three years. These differences are significant (P = 0.001, P < 0.001, and P = 0.009 at years 1, 2, and 3, respectively). Furthermore, the low risk of intervention in patients with ETDRS-DR+ results and a RETeval- DR Score is statistically indistinguishable (p ≥ 0.16) from the 8/165 (5%), 11/134 (8%), and 13/64 (20%) chance of having an ocular intervention at one, two, and three years in patients with ETDRS-DR− findings at baseline. Not shown in the figure for clarity, ETDRS-DR−/RETeval+ patients have chances of 4/24 (17%), 5/21 (24%), and 5/13 (38%) of having an ocular intervention in the subsequent one, two, and three years, which is significantly greater than the ETDRS-DR− alone at years 1 and 2 (P = 0.03, P = 0.03, and P = 0.2, respectively). Figure 5A shows a similar pattern of results for VTDR+ patients.

Figure 5.

The impact on probability of intervention after 1, 2, and 3 years of adding functional information to structural information (top row) or vice-versa (bottom row). For each year, the mean (± standard error of mean) of having a positive or negative measurement is shown for a first test. For positive first test results, the impact of adding a second test is shown. Stars represent statistically significant mean differences (without adjustment for multiple comparisons) for adding a second positive test, adding a second negative test, and between the second positive and negative tests. (A) The effect of RETeval result on prognosis in patients with VTDR+. (B) The effect of RETeval result on prognosis in patients with ETDRS-DR+. (C) The effect of VTDR result on prognosis in patients with RETeval+. (D) The effect of ETDRS result on prognosis in patients with RETeval+.

Figures 5C and 5D show the impact of structural information on the predictive value of RETeval results. In patients with RETeval+ results, there was a 12/51 (24%), 18/43 (42%), and 20/31 (65%) chance of an ocular intervention in the first 1, 2, and 3 years after testing. There is a tendency for a positive structural result to increase risk and a negative structural result to decrease risk; however, the difference in these groups is of marginal significance: 2/6 groups have P values > 0.05, 3/6 are significant at the P ≤ 0.05 level, and only 1/6 is significant at the P ≤ 0.01 level. Furthermore, RETeval− patients had lower risk of intervention than patients with RETeval +/VTDR− or RETeval +/ETDRS− DR.

All results were robust over a wide range of RETeval DR Score cutoff values (Supplementary Figures S4–S6) and for all of the individual ERG and pupillary light reflex components of this score (Supplementary Figures S1–S3), and for another ERG criterion previously published¹⁶ (Supplementary Figure S7).

Discussion

In this study charts of 252 patients with diabetes were reviewed up to three years after initial ETDRS 7-field stereo fundus imaging and evaluation of physiological measures of retinal function (ERG and pupillary light reflex). Results clearly show that baseline measures of structure and function have value in predicting intervention to treat progression of retinopathy and, importantly, that combining structural and functional information provides superior prediction of intervention than either parameter alone. Patients with structurally defined VTDR were at no greater risk of intervention than those with less-severe structural involvement if baseline visual function was not above criterion abnormality (RETeval DR Score < 23.5). In fact, the RETeval DR Score was found to indicate the highest risk of requiring ocular intervention in this study, with patients having scores ≥23.5 being 11 times more likely to have a future ocular intervention than patients having scores < 23.5.

Adding RETeval functional assessment to observations of retinal structure will improve programs designed to manage visual health in patients with diabetes. A similar concept is well established in the glaucoma literature, where it has long been recognized that combining structural measures of the integrity of the nerve fiber layer with functional results of standard automated perimetry is superior in diagnosing and staging of the disease.^21,22

Although this study used the gold standard double-read ETDRS 7-field photography, we would expect similar results using dilated fundus examinations and simpler photographic methods assuming a substantial correlation between these techniques and the gold standard.²³ With advances in technology used to obtain high quality color fundus photographs^24–26 and the use of artificial intelligence algorithms to classify these photographs,^{7,8,26,28–30} large scale programs to identify the need of patients to see a retinal specialist have been implemented to reduce morbidity associated with diabetic retinopathy.⁶ Technological advances in clinical electroretinography have also made this test much better suited to aid in the detection of diabetic retinopathy.^15–18,27 The assessment of visual function used in the current study required < 5 min of testing per patient and does not require pupil dilation,³¹ dark adaption, or eye-contacting electrodes.³² The hand-held RETeval device is a turnkey system that can be administered by a technician or medical assistant with minimal training requirements. Addition of measurement of retinal function should enhance the predictive value of AI algorithms.

A limitation of this study was the relatively small number of patients from the same hospital system in the retrospective analysis, which caused uncertainty in the RR of the prognostic criteria considered. All prognostic criteria had RRs that were statistically significantly greater than one, but the RRs among the criteria could not be differentiated with certainty. Another limitation is the absence of OCT imaging to document macular edema. Although it is likely that this would have identified more eyes at risk at baseline and additional cases of progression, it is notable that many screening programs rely exclusively on color fundus photographs. Thus the results of this study are relevant in demonstrating the benefit of adding measures of visual function to such programs. Finally, it is likely that clinical practice regarding threshold for intervention may have changed since these data were collected (2014–2017). Studies suggest that intervention may occur earlier in the course of the disease.^33,34 However, we do not think that this change in practice would impact the conclusions of this study. A strength of the study was the use of ocular interventions as the end point, because structural changes alone are imperfect in the prediction of vision loss.⁹ As illustrated in the Supplementary Material, many functional parameters can predict ocular interventions, and the best combination for predicting ocular interventions may differ from the best combination for predicting anatomic status. Further prospective studies are needed to further optimize scoring systems based on these functional tests.

To our knowledge, this is the first study to show an increase in prognostic value resulting from combining measures of retinal structure and function in diabetic patients. The American Academy of Ophthalmology recommends at least annual screening for Type II diabetics with more severe findings requiring more frequent eye exams.¹⁹ Use of teleophthalmology to identify structural findings of DR has aided in triaging diabetic patients with no or minimal findings of DR. Our results demonstrate that combining structural and functional measures of the retina has strong predictive value over two years and could shape screening guidelines further. For example, a patient with no, mild or moderate NPDR on fundus photos and a RETeval+ result might warrant intermediate ophthalmology follow-up when they would have been missed if relying on fundus photos alone (as is done currently) because our data suggest a 17% chance of needing an ocular intervention within the next year. On the other hand, a patient with severe NPDR or PDR on fundus photos and a RETeval DR− result might be spared from immediate follow-up because we found only a 3% incidence of intervention in these patients within the next year.

Improved assessment of progression risk for diabetics is beneficial to refine referral criteria so that only the highest risk patients are referred for specialized care. This is important especially in the post-pandemic healthcare era, where access to care is more challenging, and the healthcare system needs, more than ever, to dedicate its resources to the patients who are at highest risk. The improvement in referral criteria achieved by integrating structural and functional measures should result in improved patient outcomes and a reduction in the financial burden of DR management.