Measuring Quality of Life in Diabetic Retinal Disease: A Narrative Review of Available Patient-Reported Outcome Measures

Abstract

Topic

Several patient-reported outcome measures (PROMs) are available to measure health-related quality of life (HRQoL) in patients with late-stage clinical diabetic retinal diseases (DRDs). However, an understanding of the psychometric properties of PROMs is needed to assess how they could relate to severity levels of a revised DRD grading system. This narrative review assessed the available generic-, vision-, and DRD-related PROMs used in DRD research and highlights areas for improvement.

Clinical Relevance

Diabetic retinal disease is a common complication of diabetes and can lead to sight-threatening complications with a devastating effect on HRQoL.

Methods

The Quality of Life working group is one of 6 working groups organized for the DRD Staging System Update Effort, a project of the Juvenile Diabetes Research Foundation Mary Tyler Moore Vision Initiative. PubMed, Cochrane Library, Embase, and Google Scholar databases were searched using core keywords to retrieve ophthalmology-related review articles, randomized clinical trials, and prospective, observational, and cross-sectional studies in the English language. A detailed review of 12 PROMs (4 QoL questionnaires and 8 utilities) that met a minimum level of evidence (LOE) was conducted. The relevance of each PROM to DRD disease stage and Biomarker Qualification guidelines (Biomarkers, EndpointS, and other Tools) categories was also defined.

Results

The National Eye Institute 25-item Visual Function Questionnaire (NEI VFQ-25), Impact of vision impairment-computerized adaptive testing, and Diabetic Retinopathy and Macular Edema Computerized Adaptive Testing System had a LOE of II in detecting change due to late-stage DRD (diabetic macular edema), although several areas for improvement (e.g., psychometrics and generalizability) were identified. Other PROMs, particularly the utilities, had a LOE of III due to cross-sectional evidence in late-stage clinical DRD. Although the NEI VFQ-25 has been the most widely used PROM in late-stage DRD, more work is required to improve its multidimensional structure and other psychometric limitations. No PROM was deemed relevant for subclinical or early/mid-DRD.

Conclusion

This narrative review found that the most commonly used PROM is NEI VFQ-25, but none meets the ideal psychometric, responsiveness, and clinical setting digital administration requirements that could be included in an updated DRD staging system for diagnosis and monitoring of DRD progression.

Financial Disclosure(s)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Diabetic retinal disease (DRD) is a major public health burden, with clinical DRD defined as visible vascular lesions of diabetic retinopathy (DR), affecting nearly one-third of people with diabetes.¹ Classification of DRD is currently based on severity levels of clinical DR. Approximately 10% of people with DR have vision-threatening disease, specifically diabetic macular edema (DME) or proliferative diabetic retinopathy (PDR) that comprises late-stage clinical DRD. However, the current DRD classification system does not take into account the patient’s quality of life (QoL).^2,3

Patient-reported outcome measures (PROMs) are commonly used to measure health-related QoL (HRQoL). They can be generic (i.e., general health), symptom specific (e.g., vision), or disease-specific (e.g., DRD) and may be delivered as questionnaires or utility instruments (Table S1, available at www.ophthalmologyscience.org.).⁴ Questionnaires comprise a set of items (questions) whose scores can be summed or averaged to provide measurement of a latent construct (e.g., HRQoL). The overall score is typically used as a study end point or outcome measure. Utilities reflect the health status of a patient and the associated value of that health status through the patient’s preferences. Preferences can be derived directly from patient ratings of health-related hypothetical scenarios or indirectly by patients rating their health status from a multiattribute classification system (similar to a questionnaire). Utilities can be used to generate the quality-adjusted life-years (QALYs) that combines both the quantitative (length of life) and the qualitative (QoL) into a single index using a utility score, as in the equation, (years of life × utility value = QoL) years.^5,6

The inclusion of PROMs in clinical trials is now mandated by regulatory authorities, such as the Food and Drug Administration (FDA) and the European Medicine Agency.⁷ Moreover, PROMs can guide clinical care, support patient–provider decision making, provide data for comparative effectiveness research for practice improvement or assessment of performance, and assess metrics for value-based payments.⁸ Because of the importance of these parameters in health care, a better understanding of the context of use and psychometric properties of current and emerging DRD-associated PROMs is required to guide public health decisions and policies⁹ and clinical trial end point measurements.¹⁰ The aims of this paper were as follows: (1) to conduct a narrative review of the context of use, appropriateness, and psychometric properties of current and emerging HRQoL PROMs used in DRD research; and (2) to highlight gaps in knowledge that may serve as potential guidance for future work to develop and validate DRD-specific PROMs.

Methods

This narrative review was undertaken by the Quality of Life working group, one of 6 working groups organized for the DRD Staging System Update Effort, a project of the Juvenile Diabetes Research Foundation Mary Tyler Moore Vision Initiative supported by The Mary Tyler Moore and S. Robert Levine, MD Charitable Foundation (see Appendix X for a complete listing of Initiative leadership and participants, available at www.ophthalmologyscience.org). The researchers in the Quality of Life working group were led by Stela Vujosevic (Italy) and other members included Emily Chew (United States), Leanne Labriola (United States), Ecosse Lamoureux (Singapore), and Sobha Sivaprasad (United Kingdom).

Literature Search

Four databases (PubMed, Cochrane Library, Embase, and Google Scholar) were searched for QoL PROMs that have been used in DRD research using the following keywords: “Diabetic retinopathy” AND “Diabetic Macular Edema/Oedema” AND “Quality of life” AND “Functioning”; “Utility,” “Time Trade Off,” “Standard Gamble,” “Direct Elicitation,” “Health Related Quality of Life,” “ED-BOLT ON,” AND “RetCAT”. The search was completed in April 2022, and ophthalmology-related review articles, randomized clinical trials (RCTs), and prospective, observational, and cross-sectional studies were included. The review followed all 6 items according to the Scale for the Assessment of Narrative Review Articles (SANRA), an accepted critical appraisal tool to assess the quality of nonsystematic review articles.¹¹ A detailed review was completed for 12 PROMs that met the key criteria of at least level III evidence (LOE) (Table 2). These were: European Quality of Life Questionnaire (EQ-5D) and EQ-5D/visual analog scale (VAS); National Eye Institute 25-item Visual Function Questionnaire (NEI VFQ-25); Impact of Vision Impairment (IVI) questionnaire; Diabetic Retinopathy Dependent Quality of Life (RetDQoL); Diabetic Retinopathy and Macular Edema Computerized Adaptive Testing System (RetCAT); Short form-6D (SF-6D); the Health Utilities Index (HUI3); Vision and Quality of Life Index (VisQoL); Time trade-off (TTO); Standard Gamble (SG); and Diabetic Retinopathy Utility instrument (DR-U) (Table 2).

Table 2.

Currently Available Level of Evidence for the 12 Reviewed PROMs

	I	II	III	IV-V
NEI VFQ-25		Substantial evidence in RCTs and population-based studies for late-stage clinical DRD
IVI		Evidence in clinical and population-based samples
RetDQoL			CS with mostly clinical samples
DR-U			CS in a large clinical sample
RetCAT		CS in a large clinical sample
EQ-5D/VAS			CS with mostly clinical samples
SF-6D			CS with mostly clinical samples
HUI3			CS with mostly clinical samples
VisQoL			CS with mostly clinical samples
TTO/SG			CS with mostly clinical samples

CS = cross-sectional; DRD = diabetic retinal disease; DR-U = Diabetic Retinopathy Utility instrument; EQ-5D = European Quality of Life Questionnaire; HUI-3 = Health Utilities Index; IVI = Impact of Vision Impairment; NEI VFQ-25 = National Eye Institute 25-item Visual Function Questionnaire; PROM = patient-reported outcome measure; RCT = randomized controlled trial; RetCAT = Diabetic Retinopathy and Macular Edema Computerized Adaptive Test; RetDQoL = Retinopathy Dependent Quality of Life questionnaire; SF-6D = Short Form 6-D; SG = standard gamble; TTO = time trade-off; VAS = visual analog scale; VisQoL = Vision and Quality of Life index.

These PROMs’ parameter characteristics, scientific understanding of their relationship to DRD, performance expectations in regard to DRD, evidence level, statistical data, gap analysis, and additional information, such as availability and applicability (detailed in Table S3, available at www.ophthalmologyscience.org) were reviewed in a standardized manner according to the FDA Biomarker Qualification guidelines (Biomarkers, EndpointS, and other Tools, FDA 2020).¹²

This multistep review process started with an initial review by a single researcher, then proceeded with a collaborative review by the working group that met monthly between November 2020 and June 2021 and thereafter 4 more times (12 times in total) and also communicated by email until July 2022. Thereafter, the working group lead synthesized the results and formulated conclusions, which were sent to the Steering Committee.

The PROMs were classified as “Ready” (for current use or use within 2 years), “Promising” (unmet, but defined research needs that can be accomplished within 2–5 years), and “Potential” (unmet research needs that will need > 5 years to accomplish) based on time frame of anticipated validation for clinical or research use (Table 4) based on subject matter expertise and consensus agreement among all 5 members of the working group. Moreover, the relevance of the 12 PROMs to disease stage (preclinical, early, mid, and late-stage DRD) (Table 4) was also determined. Late-stage DRD was defined as PDR and/or DME. The sensitivity of each of the 12 PROMs to identify progression of DRD, responsiveness of DME and/or PDR to treatment, and visual acuity change were also considered.

Table 4.

Relevance of PROMs to Disease Stage

	Ready∗	Promising†	Potential‡
Subclinical DRD
Early-stage clinical DRD
Midstage clinical DRD
Late-stage clinical DRD	NEI VFQ-25		RetCAT; RetDQoL; EQ-5D/VAS; DR-U

DRD = diabetic retinal disease; DR-U = Diabetic Retinopathy Utility instrument; EQ-5D = European Quality of Life questionnaire; NEI VFQ-25 = National Eye Institute 25-item Visual Function Questionnaire; PROM = patient-reported outcome measure; RetCAT = Diabetic Retinopathy and Macular Edema Computerized Adaptive Test; RetDQoL = Retinopathy Dependent Quality of Life questionnaire; VAS = visual analog scale.

^∗For current use or use in < 2 years.

^†Unmet, but defined research needs that can be accomplished within 2–5 years.

^‡Unmet research needs that will need > 5 years to accomplish.

Finally, the potential relevance of the 12 PROMs was classified according to the FDA Biomarker Qualification guidelines (Biomarkers, EndpointS, and other Tools, FDA 2020)¹² (Table 5).

Table 5.

Potential Relevance of PROMs to BEST Categories

	Diagnostic	Monitoring	Predictive	Prognostic	Pharmacodynamic/Response	Safety	Susceptibility/Risk
NEI VFQ-25					X
DR-U					X
RetDQoL
IVI					X
RetCAT					X
EQ-5D
EQ-5D/VAS
SF-6D
HUI3
VisQoL
TTO
SG

BEST = Biomarkers, EndpointS, and other Tools; DR-U = Diabetic Retinopathy Utility instrument; EQ-5D = European Quality of Life Questionnaire; HUI3 = Health Utilities Index; IVI = Impact of Vision Impairment; NEI VFQ-25 = National Eye Institute 25-item Visual Function Questionnaire; PROM = patient-reported outcome measure; RetCAT = Diabetic Retinopathy and Macular Edema Computerized Adaptive Testing System; RetDQoL = Diabetic Retinopathy Dependent Quality of Life; SF-6D = Short Form-6D; SG = standard gamble; TTO = time trade-off; VAS = visual analog scale; VisQoL = Vision and Quality of Life index.

Conclusions about the LOE rating, psychometric performance, responsiveness, appropriateness, and generalizability of each PROM to detect change in DRD severity levels and treatment response informed areas for future research that are required for an ideal PROM.

Results

Questionnaires (Vision-Specific)

NEI VFQ-25

Parameter Characteristics

The NEI VFQ-25 elicits patient perceptions of their visual impairment (VI) and its relation to HRQoL.¹³ Content for the questionnaire was developed from patients with DR, age-related macular degeneration, cataract, primary open-angle glaucoma, cytomegalovirus retinitis, and low vision.¹³ The NEI VFQ-25 includes 1 general health item and 11 vision-specific domains, including general vision, ocular pain, near vision, distance vision, social function, mental health, role limitations, dependency, driving, color vision, and peripheral vision. Composite and subscale scores range from 0 to 100, where higher scores indicate better functioning or well-being.¹⁴ Although minimally important differences that correlate with severity level of clinical DR have not yet been established for the NEI VFQ-25, a ≥ 6-point change in composite or subscale scores has been reported to reflect a clinically meaningful change in DME.¹⁵ The reliability and validity of the NEI VFQ-25 has been demonstrated in a variety of eye conditions, including DRD (LOE II). For more information on NEI VFQ-25, refer to Supplementary Materials.

Scientific Understanding and Performance Expectations in DRD

The NEI VFQ-25 has shown consistent correlation with visual acuity.¹⁶ For example, in the Wisconsin Epidemiology study of Type 1 Diabetes, NEI VFQ-25 scores were strongly associated with vision, independent of severity of DR and other diabetes complications.¹⁷ The NEI VFQ-25 is also associated with severity of DR, with scores decreasing with increasing severity of DR in people with no diabetes-related comorbidities.

Type of Data and Availability for Evidential Evaluation

The NEI VFQ-25 has been used in population-based studies, case series, and RCTs, particularly with novel treatments for DME. Although the NEI VFQ-25 scores are appropriate for monitoring of vision-related QoL (VRQoL) because they are associated with vision and DR severity, their sensitivity has not been evaluated for earlier stages of DRD. It can be either self- or interviewer-administered.

Statistical Considerations

The NEI VFQ-25 has undergone substantial psychometric testing using classical test theory (e.g., construct validity, criterion validity, dimensionality via factor analysis, etc.).¹³ However, Rasch analysis of the NEI VFQ-25 has demonstrated improved psychometric performance and interpretability of the VRQoL scale using a 2-domain structure (visual functioning and socioemotional well-being).18, 19, 20 A more recent evaluation of the Rasch analysis on combined clinical trials and observational data calibrated item measures and rating category thresholds for the NEI VFQ-25 as well as for the 2 domain-specific versions, the NEI VFQ that included the visual function time and the socioemotional items. This has greatly improved the usefulness of the NEI VFQ by enabling the estimation of measures on an invariant scale and comparisons between patients and across studies.²¹ A shorter version of NEI VFQ-25 is also being developed.

Gap Analysis

Because the NEI VFQ-25 has very few items in each subscale, the results should be interpreted with caution. Moreover, it does not evaluate other aspects of VRQoL, such as functioning under low luminance. It has to be extended to more focus groups (with various severity levels of DR). It also should be considered for change into the electronic format due to the high costs in trials. The NEI VFQ-25 is not currently approved by regulatory bodies as an end point for trials. The NEI VFQ is strongly correlated with visual acuity, but it does not correlate with DRD or systemic disease. The psychometric properties of NEI VFQ-25 using modern psychometric theory has been explored. As such, revision of the NEI VFQ-25 to include more granularity or spread of the scale for each item and also to expand the number of work is still required to optimize its multidimentionality, especially in patients with DRD.18, 19, 20 As such, revision of domains may be required so it can be multidimensional and achieve the standards set out by regulatory bodies, such as the FDA and European Medicine Agency.

Miscellaneous Questions

The NEI VFQ-25 is readily available online at no cost, has been translated into multiple languages, and is quick to administer. With strong correlation with visual acuity, it merits further research and development to incorporate into the DRD staging process.

IVI

Parameter Characteristics

The 28-item IVI measures the impact of VI (from any eye condition) on VRQoL and comprises 3 domains; reading and accessing information, mobility and independence, and emotional well-being.22, 23, 24 Higher scores indicate better VRQoL outcomes and vice versa. The IVI has been extensively validated in clinical and population-based studies (LOE II),25, 26, 27, 28 including patients with DRD, and has excellent psychometric properties (LOE II).

Scientific Understanding and Performance Expectations in DRD

The IVI is correlated with visual acuity, which is reflected in its association with severity of DRD,25, 26, 27, 28 response to treatment,²⁹ and low-vision rehabilitation.³⁰

Type of Data and Availability for Evidence Evaluation

In addition to cross-sectional and population-based studies, the IVI has been used in 2 small intervention studies in Australia, including phase II RCTs in DR^29,30 in which IVI scores correlated with improvement in vision after treatment for DME.

Statistical Considerations

The IVI provides an overall score of VRQoL in addition to 3 domain scores. Scores can be calculated using simple summary scoring; however, conducting Rasch analysis on the raw data is recommended to convert the summed scores into those approximating interval-level measurement, ready for use in parametric testing such as regression analyses. Using Rasch-scaled IVI scores rather than summed IVI scores may improve measurement precision and increase sensitivity when assessing associations between variables and changes over time.³⁰ A recent study by Goldstein et al³¹ has calibrated IVI item measures from various studies, enabling researchers and clinicians to more easily compare VRQoL in patients with VI.

Gap Analysis

Although the IVI has been used in 2 phase II DRD trials to date, its responsiveness to other DRD treatment regimens has not yet been evaluated, and further work in this area is needed. It is nonspecific to DRD and provides measurement of only 3 QoL domains.

Miscellaneous Questions

The IVI is available commercially and can be administered in paper–pencil format or on a digital platform; as such, it can be easily implemented in both high and low-resource settings. It can be either self- or interviewer-administered. The IVI has relatively low-patient burden, taking approximately 15 minutes to administer. Shortened versions of the IVI are also available, including the Brief IVI,³² currently endorsed for use in assessing VRQoL in patients with age-related macular degeneration by the International Consortium for Health Outcomes Measurement and the IVI-CAT,³³ both of which cut administration time by 50% to 80%.

Questionnaires (DRD-Specific)

RetDQoL

Parameter Characteristics

The RetDQoL specifically evaluates the QoL of patients diagnosed with DR and is modeled on the Audit of Diabetes-Dependent QoL.³⁴ The questionnaire has 2 broad questions related to present QoL and patient perception of QoL if they did not have a diabetes-related eye problem. The remaining 24 items refer to specific aspects of QoL, in which patients are asked to both evaluate the impact of the domain on the QoL and how important each domain is to their life. The RetDQoL has demonstrated high internal consistency and good construct validity (LOE III).

Scientific Understanding and Performance Expectations in DRD

The RetDQoL has been shown to be sensitive to different levels of DRD-induced VI and late-stage DRD as the expected relationships were found for both the overview items and the average weighted index.³⁵

Type of Data and Availability for Evidential Evaluation

The reported use of RetDQoL in clinical practice, population-based studies, or clinical trials to assess its correlation to DR severity level or treatment outcome has been limited.^36,37

Statistical Considerations

The multiplicative scoring system employed by the RetDQoL is problematic,³⁸ as shown in previous work utilizing Rasch analysis in similar instruments (the Macular Disease Quality of Life questionnaire³⁹); the majority of response categories were underutilized and category thresholds were disordered.

Gap Analysis

The psychometric properties of the RetDQoL have not been explored using modern psychometric theory, and its responsiveness to DRD treatment regimens has not yet been evaluated.

Miscellaneous Questions

The RetDQoL is available upon request from the authors and it can be easily implemented in both high and low-resource settings. It can be either self- or interviewer-administered and is suitable for administration both face to face or via telephone (although it is recommended not to mix the mode of interview in the same study). With only 24 items, the RetDQoL is relatively quick to administer.

RetCAT

Parameter Characteristics

Diabetic Retinopathy and Macular Edema Computerized Adaptive Testing System is a digital and computerized adaptive testing (CAT) PROM that measures the impact of DRD, associated VI, and effectiveness of related treatments on QoL. Diabetic Retinopathy and Macular Edema CAT System contains 279 items within 10 QoL item banks, namely visual symptoms, activity limitation, mobility, emotional, health concerns, convenience, driving, lighting, social, and economic (Table S6, available at www.ophthalmologyscience.org). Diabetic Retinopathy and Macular Edema CAT System provides a continuous score in logits (transformed to a 1–99 percentile for better interpretability) for each QoL domain, with higher scores indicating “better” QoL outcomes. Items are administered from the item banks by the CAT system using an algorithm based on the item calibrations and patients’ responses to all previous items. Diabetic Retinopathy and Macular Edema CAT System has undergone a rigorous, multistage development and validation process (LOE II) (see Supplementary Materials).40, 41, 42, 43, 44

Scientific Understanding and Performance Expectations in DRD

With patient-involved content development, it is unsurprising that RetCAT scores are related to the severity of DRD and VI.^43,44 For instance, 4 RetCAT tests (activity limitation, health concerns, lighting, and visual symptoms) have demonstrated reductions in test scores as DRD severity increases and, for binocular VI, RetCAT scores consistently decreased as the severity of VI worsened for all domains except convenience.

Type of Data and Availability for Evidential Evaluation

To date, RetCAT has been applied in clinical, cross-sectional studies in White and Asian populations as part of its developmental processes.^40,43,44

Statistical Considerations

Unlike paper–pencil questionnaires, there is no need to treat RetCAT data with modern psychometric methods because the scores have been generated from a Rasch-calibrated item bank.

Gap Analysis

Responsiveness to treatment options and psychometric performance in DRD progression need to be further explored by its use in RCTs, clinical and population-based studies, and health care systems.

Miscellaneous Questions

Diabetic Retinopathy and Macular Edema CAT System is ready for use and commercially available. It provides fast measurement of QoL (∼2 minutes per QoL domain) while maintaining high measurement precision.⁴³ With 10 domains available, it provides more comprehensive measurement than 2 to 3 domain paper–pencil questionnaires. Because RetCAT requires internet access and computer hardware (e.g., laptop and tablet) for administration, it may be less accessible for low-resource settings. However, for high resource settings, the RetCAT system reduces data storage, entry, and analysis needs, and enables real-time feedback via integration with e-health record systems. Diabetic Retinopathy and Macular Edema CAT System can be administered in clinic or at home via a secure URL on patients’ own smart devices, making it attractive for health care in the coronavirus disease 2019 era where patients are spending minimal time physically attending clinical appointments. Qualification from the FDA is currently being sought for RetCAT.

Utilities (Generic)

EQ-5D and EQ-5D/VAS

Parameter Characteristics

The EQ-5D is a multiattribute utility instrument that measures generic HRQoL.⁴⁵ It has 5 health dimensions (mobility, self-care, usual activities, pain/discomfort, and depression/anxiety).⁴⁶ Each dimension has either 3 levels of severity (EQ-5D-3L, describing 243 health states) or 5 levels of severity (EQ-5D-5L, describing 3125 health states). Health status can also be valued directly using the EQ-5D/VAS ranging from 0 to 100, with 0 representing the worst imaginable health state and 100 the best.⁴⁷

Scientific Understanding and Performance Expectations in DRD

In health technology assessment, a difference of 0.06 to 0.07 points is considered a clinically meaningful change in DME in the better seeing eye. Change in EQ-5D does not correlate with changes in visual acuity outcomes in clinical trials evaluating interventions for DME.^6,48, 49, 50 Similarly, the EQ-5D has been unable to elicit a difference between DR severity levels classified according to the better eye, worse eye, or VI in the worse eye in cross-sectional studies.^51,52 The disutility of blindness in one eye ranged from −0.074 to −0.054.^53,54 Only extreme visual acuity categories demonstrate a statistically significant difference (LOE III).

Type of Data and Availability for Evidential Evaluation

The EQ-5D has been used in cross-sectional, clinical, population-based, and RCT studies in patients with DR.^51,55,56

Statistical Considerations

The EQ-5D-3L index ranges from −0.039 to 1,4, 5, 6 with scores derived from responses from 3395 people randomly selected from the general population in the United Kingdom.⁴⁷ Value sets from a range of different countries are available for the EQ-5D-3L and EQ-5D-5L, with cross-walk value sets also available to map 3L scores on 5L scores.

Gap Analysis

Overall, studies investigating the validity and reliability of EQ-5D in various severity levels of DR are limited, and there is a lack of longitudinal validity to measure a change in health-related utility based on DR over time.^51,52 Changes in EQ-5D may also reflect the feelings of vulnerability to vision loss in patients with mild DR, which confound the values.⁵⁷ Similarly, the EQ-5D may capture changes related to side effects of potential treatments of DR in the context of general health dimensions of an individual.

To address the lack of sensitivity of the EQ-5D to vision loss, a set of vision questions have been added as a supplemental bolt-on,⁵⁸ although this has not been validated against severity levels of DR.⁵⁹ Moreover, because visual acuity is unaffected even in high-risk PDR without complications, the EQ-5D bolt-on is also unlikely to capture changes that correlate with changes in DR severity levels.

Miscellaneous Questions

Despite the gaps reported above, the EQ-5D is the most used generic preference-based measure of health in ophthalmology and is quick to use and freely available.

Although the EQ-5D is useful to define the impact of ocular disease compared with other health parameters, it has limited usefulness as a biomarker of disease-specific vision deterioration. Additional modifications to the scale of EQ-5D that can account for responsiveness to change in the DRD staging system may be worthwhile because it may serve as a comparator when evaluating other ranking or discrete choice methods because these methods may also correlate with other systemic comorbidities in people with diabetes and help decipher the findings observed in other methods.

SF-6D

Parameter Characteristics

The SF-6D is derived from the SF-36 and SF-12 health questionnaires and has 6 dimensions (physical functioning, role limitation, social functioning, bodily pain, mental health, and vitality), and each dimension has 4 to 6 severity levels.^47,60 This results in 18 000 possible unique health states, which is higher than the EQ-5D and therefore may mean the SF-6D can detect more subtle changes in health state than the EQ-5D. A tariff of values for each health state is also available for several countries.

Scientific Understanding and Performance Expectations in DRD

In a cohort study on people with type 2 diabetes with 90% having ≥ 2 comorbidities over 1 year, both the EQ-5D and SF-6D performed equally well in people with well controlled diabetes but differed in people with multiple systemic comorbidities (including 38% with an eye disease).⁶¹ There are no data on the performance of the SF-6D based on severity levels of DR or its correlation with changes in DR severity level (LOE III).⁶²

Type of Data and Availability for Evidence Evaluation

The SF-6D has been compared with EQ-5D and -15D utilities in a cross-sectional study on type 2 diabetes with and without comorbidities and complications, including DR.⁵⁶ The SF-6D utilities were not sensitive to differentiate people with and without DR.

Statistical Considerations

The theoretical range of SF-6D preference-based utility scores based on the SG technique ranges from 1 for full health to 0.345 for the worst possible health state. Unlike the EQ-5D, the SF-6D includes role limitation and social functioning, meaning it provides a more holistic measure of HRQoL. Any patient who completes the SF-36 or the SF-12 can be uniquely classified according to the SF-6D. However, while the EQ-5D suffers from a ceiling effect, the SF-6D has a floor effect,⁵⁵ meaning that it lacks capability to differentiate HRQoL in those at the lower end of the “ability” spectrum.

Gap Analysis

There is no evidence that the SF-6D can accurately correlate with DRD.⁵⁵

Miscellaneous Questions

The SF-6D is quick to use, freely available, and can be easily calculated from its parent PROMs, SF-12 or SF-36, making it a flexible PROM option.

Utilities (Vision-Specific)

HUI3

Parameter Characteristics

The HUI3 has 9 dimensions (vision, hearing, speech, ambulation, dexterity, emotion, cognition, self-care, and pain), and each dimension has 5 or 6 severity levels, resulting in 972 000 possible unique health states. Compared with other generic preference-based measures, it also has a vision dimension.^48,49

Scientific Understanding and Performance Expectations in DRD

Using TTO to generate utilities from the HUI3, mean values of disutility in no DR, background DR, PDR, and legal blindness were 0.94, 0.87, 0.83, and 0.81, respectively, with a standard deviation of 0.14.⁶³ When considering DR severity in the worse eye, the mean QALY weights for no DR, background DR, PDR, DME, and legal blindness were 0.88, 0.78, 0.81, 0.79, and 0.39, respectively. The HUI3 showed a high correlation with visual acuity and with the NEI VFQ-25.⁵⁵ A decrement of the HUI3 of 0.057 has also been reported in those with DR and severe visual loss.⁶⁴ However, the influence of severity levels of DR was not assessed (LOE III).

Type of Data and Availability for Evidence Evaluation

The HUI3 has been used within the ACCORD clinical trial as a PROM context to measure QALYs and cardiovascular event-free years gained.⁵⁵ It relates to VI better than EQ-5D, but it was not sensitive enough to identify severity levels of DR unless there was associated VI.

Statistical Considerations

The value of the HUI3 multiattribute utility score ranges from −0.36 to 1, where 1 represents perfect health, 0 represents death, and a negative index score indicates a health state considered to be worse than death. These values were elicited from a random sample of 504 people in a Canadian general population using the VAS in combination with the SG method.^47,61,62

Gap Analysis

Overall, studies investigating the association of HUI3 utilities to severity levels of DR are limited, and there is a lack of longitudinal data to measure a change in utility based on DR progression over time. Future research is required to understand whether it is worthwhile to be incorporated in the DRD staging system.

Miscellaneous Questions

The HUI3 requires a license to use, with an associated cost. With multiple domains and items, it is relatively lengthy to administer, and the scoring algorithm to convert raw scores to utilities is relatively complex.

VisQoL

Parameter Characteristics

The VisQoL is a descriptive system derived from the Assessment of QoL-7D that covers 6 dimensions of self-reported VRQoL: physical well-being, independence, social well-being, self-actualization, planning, and organization.⁶⁵ The health states defined by the VisQoL responses are translated into VisQoL utilities using an available value set that was derived from participant surveys of the general population using the TTO method.

Scientific Understanding and Performance Expectations in DRD

Vision and Quality of Life Index changes correlated with very severe visual loss in DME and were independently associated with VisQoL utilities (LOE III).⁶⁶

Types of Data and Availability for Evidence Evaluation

The VisQoL has been utilized only in cross-sectional studies related to DR/DME.

Statistical Considerations

Item utilities are combined using a multiplicative model and the scale of the utility index ranges from −0.25 to 1, where 0.0 represents death, 1.0 represents full health, and −0.25 represents worse than death.⁶⁷ The VisQoL utilities are categorized into 5 groups using 15th, 30th, 45th, 60th, and 75th percentiles as cut points ranging from poorest to highest VRQoL. Regression coefficients for each group are provided and represent the change in VisQoL utility score per unit change of each covariate for that particular level.

It should be noted that the predicted utility score may be less accurate for application at the individual level. However, the highly accurate group-level predictions ensure the usefulness of mapping algorithms in the cost utility analysis.

Gap Analysis

The VisQoL has been evaluated only in cross-sectional studies, and criterion and convergent validity have not been established.

Miscellaneous Questions

The VisQoL is a freely available, noninvasive test that takes < 10 minutes to complete. It can be either self- or interviewer-administered.

Utilities (DRD-Specific)

TTO and SG

Parameter Characteristics

Time trade-off and SG are both direct elicitation methods that measure the utility scale and are a currently accepted model for QoL measurements in ophthalmology (Table S7, available at www.ophthalmologyscience.org). A utility value of 1.0 is equivalent to perfect health. A utility value of 0.0 is equal to death. Negative values represent a state worse than death.⁶⁸ In SG, a subject is given the scenario that a treatment, when successful, will provide a perfect health state, but an unsuccessful treatment will result in immediate death. The subject is then asked what percentage chance of treatment failure they would accept before agreeing to the treatment.⁶⁸

Scientific Understanding and Performance Expectations in DRD

Values have been established for different levels of visual acuity using TTO and SG, although values vary considerably (Supplementary Materials). For example, for 20/40 vision, utility scores of 0.80 (TTO) and 0.89 (SG) have been established, whereas for 20/400 vision, scores are 0.54 (TTO) and 0.59 (SG). For no-light perception, utility scores are 0.35 (TTO) and 0.49 (SG). Because of their numeric scale, utility scores can be compared with other health disease states⁴⁸ (e.g., major stroke [0.61]⁶⁹ and rheumatoid arthritis [0.77]⁷⁰) that have also been scored on the TTO scale. This comparative capability is advantageous to other methods of utility measurements,⁵² as being able to compare disease states across different conditions is a critical requirement for informing public health decisions.

Useful cut points with defined values have been proposed for visual acuity for both TTO and SG and for TTO for severity of DR (LOE III).⁶

Types of Data and Availability for Evidence Evaluation

Brown et al⁶⁸ established the landmark publication for TTO values that were assessed in 325 subjects (0.77, standard deviation = 0.23, 95% confidence interval = 0.02), and these values are most often used today. Since then, other groups in the United States, Canada,⁷¹ Iran,⁷² Thailand,⁷³ Sweden, Taiwan,⁷⁴ and China⁷⁵ have also utilized TTO questions to establish utility scores for patients with DR.⁶

Statistical Considerations

Covariables sex and age have been previously shown to have an effect on HRQoL⁵² and therefore should be adjusted for when analyzing utility scores. Duration of disease may also impact TTO utility ratings scores, and utility scores have been shown to shift over time, suggesting that patients with vision loss adjust to living with the disability.^49,63,76 The reliability of TTO/SG is affected by variability in the process (style of questioning and choice of words⁷⁷) as well as in the patients (cultural variations,⁷⁵ comfort with discussing death,⁷⁵ degree of cognitive loss,⁷⁸ duration of disease,⁷⁹ current comorbilities,⁸⁰ perception of their own life compared with others, opinion of ideal health,⁸¹ and past experience with health care system^81,82). Other critical limitations include mathematical inconsistences that compromise the validity of the equation and participants’ rating of worse than death.⁸¹

From a statistical standpoint, utility scales are interval scales (where 0 is an arbitrary value for death). This number is then entered a ratio-scale with a true zero. This violates the mathematical principles of use of interval numbers and compromises the validity of the equation.

Gap Analysis

The pivotal TTO studies for DR had small sample sizes and lacked diversity, which limits their applicablilty in larger scale studies. Additionally, although one study showed good retest reliability⁷⁹ with TTO in patients with ocular disease, several other studies highlight reliability as a limitation of this method.^71,76,83,84 The sensitivity of TTO/SG utility scores to detect progression of DRD is not known,⁷² and additional information on the ability of this tool to measure change after intervention is needed.

Miscellaneous Questions

Overall, evaluation of TTO/SG utility scores is highly useful in comparing DRD to other disease states. However, the lack of testing standardization and the variable cultural acceptance of this form of questioning creates significant challenges for use of this measure as a robust biomarker for QoL. Other methods of eliciting utilities, such as ranking methods and discrete choice experiments (DCEs) have been shown to have higher reproducibility and reliability.

DR-U

Parameter Characteristics

The DR-U is a preference-based utility instrument developed using a DCE in a clinical sample of 220 patients with DR/DME across the spectrum of disease severity.⁸⁵ Utilities are estimated from the DR-U via a descriptive system comprising 5 dimensions of DR-specific QoL, including visual symptoms, lighting and glare, activity limitation and mobility, socioemotional well-being, and inconvenience, with each dimension rated using 3 severity levels ranging from “no difficulty” to “a lot of difficulty.”

Scientific Understanding and Performance Expectations in DRD

The DR-U is sensitive to DR severity and DR-related VI, with utilities decreasing as DR severity (nonvision-threatening DR = 0.87; vision-threatening DR = 0.80; P = 0.021) and better eye VI (none = 0.84; mild = 0.78; moderate/severe = 0.72; P = 0.012) increased. The DR-U also demonstrated divergent and convergent validity, with low (r = 0.39) and moderate (r = 0.58) correlations with EQ-5D and VisQoL utilities, respectively (LOE III).⁷

Type of Data and Availability for Evidence Evaluation

The use of the DR-U beyond the development population (cross-sectional) is limited.

Statistical Considerations

The value of the DR-U utility ranges from 0.58 to 1, where 1 and 0.58 represent the best and worst possible health states, respectively. The bottom anchor was rescaled from 0 to 0.58 based on the mean TTO value for DR-related blindness in 7 reported studies.⁵² The raw scores collected using the DR-U descriptive system are converted to utilities using the utility weights determined by the DCE. The scale of the DR-U differs from the “full health” and “being dead” QALY scale, meaning that comparisons across different diseases and treatments on a common metric and generation of QALYs are difficult. Studies that intend to inform decision making on a general population level may also need to incorporate a generic utility instrument.

Gap Analysis

The DR-U is not approved by regulatory bodies. Discrete choice experiments use a flexible methodology to estimate which attributes are important in decision making.⁸⁶ The DR-U’s responsiveness to change over time due to intervention and its association with all levels of DRD remain unclear as does the applicability and reproducibility of this measure across a diverse population.

Miscellaneous Questions

Because the DR-U is commercially available in both paper–pencil and digital format, it can be easily implemented in both high and low-resource settings. It can be either self- or interviewer-administered. The DR-U takes approximately 5 to 10 minutes to administer. Qualification from the FDA is currently being sought for the DR-U.

Summary of Key Findings

Of the 12 reviewed PROMs, the NEI VFQ-25 had a LOE II and a wide usage in cross-sectional, longitudinal, and RCT studies (Table 2). The IVI and RetCAT were given a LOE of II due to their large-scale application in both clinical and population-based studies and robust development and validation protocol, respectively. Most PROMs, particularly the utilities, had a LOE of III due to most of the evidence being cross-sectional in nature. The NEI VFQ-25 is ready to be included in late-stage DRD, mainly in DME (Table 4), whereas EQ-5D/VAS, RetCAT, RetDQoL, and DR-U are considered potential for late-stage clinical DRD. No PROMs were deemed currently relevant to subclinical DRD or early stages of clinical DRD.

Discussion

In our critical review of the available generic-, vision-, and DRD-related questionnaires and utility instruments, we showed that NEI VFQ-25, EQ-5D/VAS, RetCAT, RetDQoL, and DR-U have the potential to be included in an updated DRD staging system, provided further satisfactory modifications and validation of these tools meet the required LOE. Further areas of focus for these measures include the ability to elicit a difference between different severity levels of clinical DRD and demonstrate longitudinal validity in terms of responsiveness to DRD interventions. A minimal identifiable difference or a clinically meaningful change as a response to interventions needs to be identified, especially if the worse eye is being treated. Moreover, these PROMs will need further development and validation if new DRD staging is developed that includes preclinical or early structural or functional changes that precede visual acuity changes.⁸⁷

A key advantage of the NEI VFQ-25 is that it has been widely used in DRD research, and therefore there is much comparative data available. However, its psychometric limitations and optimal functionality must be taken into consideration if it is to be further used in future DRD staging. Similarly, the lack of cut-off scores limits its usefulness in DRD staging and points to an area for future research.

Importantly, there is also an unmet need to move from paper–pencil questionnaires to computerized systems in the digital and post-coronavirus disease 2019 era. One potential solution is item banking and CAT. Computerized adaptive testing is a digital method of administering items (questions) from calibrated item banks. Each item bank measures a latent construct (e.g., “activity limitation”) and usually contains more items than normally found in a paper–pencil questionnaire. Using an algorithm based on artificial intelligence, CAT iteratively selects an item from the bank that most closely targets the person’s level of the construct at that point in the test. This “smart-technology” means that scores are generated quickly using only a subset of the available items. To our knowledge, the RetCAT, reviewed in this paper, is the first CAT in DRD available for clinical, research, and health care needs.

Our review of generic multiattribute utility instruments, such as SF-6D, SF-12D, and HUI3, suggests that they require significantly more research to understand whether any of them can be incorporated into the DRD staging system because they lack sensitivity to DRD and related clinical outcomes. Similarly, the TTO and SG have several psychometric limitations, including flawed assumptions of the model, methodological and cultural issues, and response shift.⁸¹

An alternative approach would be to use DCE and ranking methods.⁸² Such methods have shown promise for health state valuation as they offer a flexible methodology to estimate which attributes are important in decision making. However, these methods are also affected by participants drawing on past experiences and taking short cuts in their decision making.⁸⁸ The DR-U is the only currently available DRD-specific utility measure that has been developed using the DCE method. Future studies to determine its responsiveness to intervention/change over time, the detailed relationship with DRD levels, and to establish valid cut points and assess its cross-cultural applicability would strengthen the robustness of evidence of the DR-U’s measurement properties.

Strengths of this study include the use of validated and comparative frameworks (i.e., LOE and Biomarkers, EndpointS, and other Tools) to assess the 12 chosen PROMs, which added robustness and objectivity of the review process. The main limitation of the review process is the potential for bias in terms of which PROMs were selected for review, given that this was not a formal, systematic review nor a full scoping review process but simply a narrative review based on the perspectives of a limited group of PROM and clinical experts. However, tiered evaluation by this group of experts should have mitigated the likelihood for inconsistencies or controversial assessments.

Current DRD-related PROMs face a major challenge in that early and midstages of clinical DRD may not affect visual acuity, which primarily impacts patient’s HRQoL. Unless other visual function parameters correlate better than visual acuity in detecting change in early DRD, it is challenging to meet the standards of an ideal PROM. Consequently, the ultimate PROM for DRD, potentially using a CAT system, needs to be further validated using both classical and modern psychometric methods in a large sample of patients across the spectrum of DRD, with a proven capacity to detect disease progression through all clinical DRD stages and respond to interventions at any DRD severity level. Digital administration options of this ideal DRD PROM are needed, especially via tablets and URLs sent to patients’ own smart devices. However, digital exclusions and disparities need to be considered. An ability for time-effective and streamlined PROMs reports to be available in real-time, and even routed to health care electronic medical record systems to facilitate doctor–patient DRD management discussions, is ideally sought.

In conclusion, this narrative review has determined that no current PROM has all necessary parameters to be included in the updated DRD staging system. Collaborative efforts involving clinicians, trialists, modelers, economists, patient focus groups, and regulatory and funding agencies (e.g., FDA, Patient-Centered Outcomes Research Institute, NEI, European Medicine Agency, and National Institute for Health and Care Excellence) are being planned to develop and validate PROMs. Novel digital solutions (i.e., virtual reality), changes to existing validated PROM questionnaires, and application of utilities to patient-centered outcomes are some of the areas of key interest.

Manuscript no. XOPS-D-23-00087R1.