Responsiveness of the EQ-5D to the Effects of Low Vision Rehabilitation

Abstract

Purpose.

This study is an evaluation of the responsiveness of preference-based outcome measures to the effects of low vision rehabilitation (LVR). It assesses LVR-related changes in EQ-5D utilities in patients who exhibit changes in Activity Inventory (AI) measures of visual ability.

Methods.

Telephone interviews were conducted on 77 low vision patients out of a total of 764 patients in the parent study of “usual care” in LVR. AI results were filtered for each patient to include only goals and tasks that would be targeted by LVR.

Results.

The EQ-5D utilities have weak correlations with all AI measures, but correlate best with AI goal scores at baseline (r = 0.48). Baseline goal scores are approximately normally distributed for the AI, but EQ-5D utilities at baseline are skewed toward the ceiling (median = 0.77). Effect size for EQ-5D utility change scores from pre- to post-LVR was not significantly different from 0. The AI visual function ability change scores corresponded to a moderate effect size for all functional domains and a large effect size for visual ability measures estimated from AI goal ratings.

Conclusions.

This study found that the EQ-5D is unresponsive as an outcome measure for LVR and has poor sensitivity for discriminating low vision patients with different levels of ability.

The recent shift toward outcomes-based reimbursement and patient-reported outcome measures in research has led to the development of new instruments that evaluate vision-specific (and other disease-specific) quality of life outcomes. These instruments include the NEI-VFQ¹, the LVQoL² and the VisQol³,among others. Additionally, the US Panel on Cost Effectiveness in Health and Medicine⁴, as well as other governmental agencies in the US and the UK^5,6, has advocated for the use of preference-based outcome measures when evaluating interventions in research studies, clinical trials and clinical practice. Because the panel recommends outcome measures that can be used across different diseases and in different areas of medicine, it is essential to understand how these more generic preference-based outcome measures perform in patients with vision conditions. Specifically, it is important to understand how responsive these measures are when the interventions are rehabilitative rather than curative, as is the case of low vision rehabilitation. In order for preference-based outcome measures to be useful (whether general or condition specific), they must be responsive to clinically meaningful effects of the interventions being compared.

Preference-based outcome measures attempt to quantify gains and losses in health-related quality of life in terms of changes in Quality-Adjusted Life Years (QALYs). QALYs depend on health utility measures that theoretically quantify the same variable for any type of disease state or intervention.⁷ Utilities are defined as the estimated value of a current health state, expressed relative to the risk of death a person is willing to accept to achieve a perfect health state.^8,9,10 Utilities can range from 0 (value of a health state that is equivalent to death) to 1(value of a perfect health state). Since QALYs are simply the product of remaining years of life and the average health utility over those years, QALYs include both morbidity and mortality in a single variable.⁷

Estimates of health utilities are obtained with a variety of methods, but the Standard Gamble (SG) and Time-Trade Off (TTO) techniques have been used the most.¹¹ The SG method for estimating utilities requires subjects to report the level of risk of immediate death that they are willing to accept in order to achieve a perfect health state, for example, surgery that carries a specified immediate mortality risk but promises perfect health if the patient survives. One of the main differences between the SG method and the TTO method is that the TTO method deals with a future risk of premature death, rather than an immediate risk. Subjects are asked how many years of their future life they would trade off to know that their remaining years would be spent in perfect health. Although in theory the TTO should provide the same utility estimate as provided by SG, studies have shown that in practice there is disagreement between SG and TTO-estimated utilities from the same subjects.¹²

In the case of vision disorders, the risk of immediate death is an abstract hypothetical because it rarely applies to current interventions. Also, in the mind of the patient, vision impairment often is not equated with health state. For this reason, perfect vision has sometimes been used as the promised outcome, rather than perfect health, and blindness has been substituted for death, which has led to considerable controversy.^9,13

An alternative approach to estimating quality of life is to employ rating scale questionnaires such as the WHOQOL¹⁴, SF-36¹⁵, HUI Mark3¹⁶, and EuroQoL.¹⁷ Some of these questionnaires, or subsets of items in the questionnaires, have been calibrated against TTO measures of utilities obtained from the same respondents.¹⁶ Thus, patterns of responses to the items in the questionnaires can be mapped to TTO-estimated health utility values.¹⁸ The EuroQoL (EQ-5D) in particular has been standardized as an alternative to SG and TTO methods for estimating health utilities.¹⁷ In addition, the EuroQoL is recommended by a number of agencies both in the US and in the UK and, as reviewed by Tosh et al.,¹⁹ it is widely used in ophthalmic outcome studies.¹⁹ It has been recommended by the U.S. Public Health Service (PHS) Panel on Cost-Effectiveness in Health and Medicine,⁴ as well as the National Institute for Clinical Excellence (NICE) in the UK²⁰ and a National Health Service (NHS) Task Group.⁶ It has also been advocated by the Board on Healthcare Services (BHS) as described by the National Academies Press (NAP) in Valuing Health for Regulatory Cost-Effectiveness Analysis.⁵

INSTRUMENTS

EQ-5D

The 5 items in the EQ-5D represent 5 health domains with 3 possible health states for each domain. Therefore, there are 3⁵=243 possible combinations of item responses that theoretically correspond to 243 unique health states, each having its own health utility value. Three of the domains have response categories that relate directly to function (Mobility, Self-Care, and Usual Activities), two of the domains describe feelings (Pain/Discomfort and Anxiety/Depression).

The EQ-5D index (or national tariff) maps all possible combinations of responses to utilities giving a single index to compare across studies and disease states.²¹ The mappings were developed by administering the TTO for different EQ-5D item response patterns, out of the possible 243 patterns, to the general population, rather than to patients with specific diseases. Once utilities were estimated from TTO responses for different combinations of item responses, an algorithm was developed (using multi-attribute utility theory) that could generate a utility for any possible set of responses to the EQ-5D.²² The EQ-5D was developed with the explicit goal of providing a general measure of health state and of functioning as an adjunct outcome measure in clinical studies.²¹ The utility estimated from EQ-5D responses then can be used to calculate QALYs gained by a specific intervention; this approach greatly simplifies cost-effectiveness research.²²

The EQ-5D tariff was developed and calibrated first in the UK; it was later administered to a large community-based sample in the US to calibrate the instrument for the US population.¹⁷ The EQ-5D has since been used in many research studies to estimate health utilities because of the ease of administration, as well as the ability to compare different diseases on the same scale.

Low vision rehabilitation (LVR) is expected to change components of a person’s quality of life by improving that person’s ability to perform everyday activities. Daily activities are often limited by more than just vision and goal-driven LVR is concerned with restoring the person’s ability to perform the activities, independent of the person’s health state or the severity of the visual impairment. However, visual impairments rarely are accompanied by pain, and although the prevalence of depression is reported to be higher in patients with disabling visual impairments²³, altering psychological state is not an explicit goal of low vision rehabilitation in the United States. Thus, because of the coarseness of the response categories and the limited number of responses relevant to low vision, we might expect the EQ-5D to have limited sensitivity to changes in quality of life in low vision patients.

The Activity Inventory (AI)

The AI²⁴ was developed as an outcome measure specific to LVR. The AI is used adaptively — patients are asked to rate the difficulty of only those items that are important and relevant to them²⁴; this feature is relatively unique among quality of life and functional measures. There are a total of 510 items in the AI item bank including 50 goals and 460 tasks nested under the goals. The AI is designed to measure the effect of LVR on the tasks and goals that are most relevant to a patient. Because Rasch analysis is employed to estimate measures from the difficulty ratings, missing data resulting from the use of an adaptive algorithm does not pose a problem – they affect measurement precision, not measurement accuracy.²⁴

This study evaluates how well the EQ-5D can discriminate between low vision patients with different levels of vision disability, as defined by the AI, and whether it can function as a sensitive outcome measure of intervention effects meaningful to the patient that are measured with the AI. Understanding how the EQ-5D responds to vision rehabilitation relative to the benchmark set by the AI will facilitate clinically meaningful cross-disciplinary comparisons of intervention as recommended by the US Panel on Cost Effectiveness.⁴

METHODS

Data were collected from 2008 to 2011 as a part of a larger study designed to characterize the baseline traits of low vision patients who present for low vision rehabilitation (LVR) at existing private outpatient centers and to measure the effectiveness of usual low vision rehabilitation services.²⁵

Subjects

In the parent study, baseline telephone interviews were conducted on 764 patients prior to their initial visit at one of 28 LVR centers across the United States, which included both private practices and hospital-based outpatient rehabilitation clinics.

Table 1 compares the distributions of patient traits for the EQ-5D subset to the distributions of patient traits in the parent study. The trait distributions were similar for the two groups with a slightly greater percentage of severe to profound visual impairment in the EQ-5D subset. The most common diagnosis of patients in the study was macular disease. Non-neovascular macular degeneration accounted for 26% of patients, neovascular disease was 12% and other macular disorders made up 17% of patients, totaling 55% of patients in the study who presented with macular disorders. (Full details regarding demographics are available in the paper describing baseline traits of the population in the parent study).²⁵

Table 1.

Comparison of patient traits in parent study and EQ-5D Subset.

Patient Characteristics	Parent Study	EQ-5D Subset
Age Range	19–98	27–98
Median Age	77 years	77 years
% over age 65	75%	80%
%Female	66%	71%
Profound impairment (20/500 or worse)	6%	10%
Severe Impariment (20/200–20/500)	19%	26%
Moderate Impairment (20/70–20/200)	38%	34%
Mild Impairment (better than 20/70)	37%	30%

The study was an observational study which evaluated the effects of “usual care” so there was no specific treatment protocol. Usual care was determined by each clinical site but generally includes a trial frame refraction, near device assessment and may include contrast sensitivity testing and/or visual field testing; some sites performed microperimetry. All sites included training with low vision devices, whether as part of occupational therapy intervention or through services provided by a vision rehabilitation therapist. Each site had its own low vision protocol and catered the exam and rehabilitation process to the specific patient as is traditionally done at low vision centers throughout the United States. Patients were interviewed by telephone prior to their first visit at an LVR site and then they were interviewed again 6–9 months after their initial interview. Inclusion criteria were: age 18 or older, new patients to the LVR center (i.e. no services from the clinician or center for greater than 3 years), and ability to hear, understand and respond to questions in English over the telephone.

The study was approved by The Johns Hopkins University Institutional Review Board and adhered to the tenets of the Declaration of Helsinki. In addition, all the study sites complied with the requirements of the Health Insurance Portability and Accountability Act and, when required, also obtained separate institutional review board approval for their participating centers. All patients gave oral consent for their study participation before enrollment.

Procedures

Patients were administered multiple surveys by telephone interview. An intake survey which obtained detailed ocular, medical, physical, psychological and social history information was administered to all participants at baseline. The decision to obtain EQ-5D outcome data was made late in the study. The EQ-5D was administered to 77 consecutive participants (of the total 764 subjects) at baseline and again at follow-up 6 to 9 months later. These subjects closely match the subjects in the overall parent study. (See Table 1) The EQ-5D was scored by transforming subjects’ responses to utilities using the US index developed by Shaw, et al.²²

Additionally, the Activity Inventory was administered during the baseline and follow-up telephone interviews for these 77 participants. The Activity Inventory (AI) item-responses were filtered before being analyzed. Filtering removes items that describe activities that do not require intervention because they are “not difficult” or not important or relevant for the participant at baseline. The filtered AI is designed to measure the average effectiveness of rehabilitation across targeted activities that make up rehabilitation goals.²⁶

RESULTS

Rasch analysis with the Andrich rating scale model²⁷ was used to estimate overall visual ability for each participant from their difficulty ratings of AI goal items and to estimate functional ability in four domains (reading, mobility, visual information processing, and visual motor function) from difficulty ratings of corresponding subsets of AI task items. The different ability measures were estimated from goal and task difficulty ratings using Winsteps ²⁸ (with item measures and response category thresholds anchored to calibrated values estimated from the responses of over 3400 low vision patients at pre-rehabilitation baseline). For both pre-rehabilitation ability measures at baseline and post-rehabilitation ability measures at 6 to 9 months follow-up, goal and task items were not included in the analysis if the participant responded that the item was “not difficult” at baseline.

The correlation matrix in Table 2 displays Pearson correlations between baseline ability measures for the various domains of the AI and the EQ-5D baseline utilities. The EQ-5D correlates best with the AI goals score at baseline (r=0.48), but correlations with other AI functional domains are relatively weak (0.16 ≤ r ≤ 0.32), especially when compared to the correlations between these domains (0.50 ≤ r ≤ 0.83).

Table 2.

Baseline Correlations of AI Domains as Compared to EQ-5D Utility Estimates.

N =77	Reading	Mobility	Vis Motor	Vis Info	Goals	Utility
Reading	1
Mobility	0.58	1
Vis Motor	0.82	0.60	1
Vis Info	0.69	0.60	0.75	1
Goals	0.76	0.50	0.83	0.78	1
Utility	0.16	0.23	0.32	0.21	0.48	1

Figures 1 and 2 are histograms of the distribution of the baseline utilities estimated from EQ-5D responses (Figure 1) and the distribution of baseline visual ability estimated from AI goal difficulty ratings (Figure 2).

Figure 1.

Histogram illustrating the distribution of the baseline utilities estimated from EQ-5D responses obtained prior to LVR services.

Figure 2.

Histogram illustrating the distribution of the baseline visual ability in logits estimated from difficulty ratings of AI goals prior to LVR services.

The distribution of baseline EQ-5D utilities is skewed toward the ceiling of 1.0 with the mode at 0.8, the median at 0.77, and the mean at 0.74. In contrast, baseline goal scores from the AI are more normally distributed in terms of visual ability with the mean at 0.63, the median at 0.64, and the mode at 0.25 to 0.75 logit.

Measures of the effects of intervention are presented as change scores, i.e., the difference between measures estimated from follow-up responses and measures estimated from baseline responses. As illustrated in Figure 3, the distribution of EQ-5D utility change scores is approximately normal with a mean of 0.009 and a standard deviation of 0.23 (inset in Figure 3 shows a linear relationship between z-score and change in utility, with an R² of .985).

Figure 3.

Histogram illustrating distribution of EQ-5D utility change scores

Figure 4 illustrates that the distribution of visual ability change scores (estimated from AI goal responses) also is approximately normal with a mean of 0.89 and a standard deviation of 1.28 (inset in Figure 4 shows a linear relationship between z-score and change in visual ability, with an R² of .963). The Pearson correlation between utility and visual ability change scores is 0.056, which is not significantly different from zero (p=0.32).

Figure 4.

Histogram illustrating distribution of visual ability change scores estimated from difficulty ratings of AI goals.

Although EQ-5D utility change scores and AI functional domain change scores are in different units, effect sizes can be compared using Cohen’s effect size index (d = average change score/baseline standard deviation), as shown in Figure 5. The effect size of LVR was calculated for the AI domains as well as for the EQ-5D utilities. Effect sizes in the range of 0.2–0.3 are considered small, effect sizes around 0.5 are considered medium, and effect sizes of 0.8 or greater are considered large. As illustrated with bar graphs in figure 5, there are medium size effects for most AI functional domains and a large effect for visual ability (based on goal ratings). The effect for utilities based on EQ-5D response is not significantly different from zero.

Figure 5.

Comparison of Cohen’s effect sizes for EQ-5D utility change scores, visual functional ability change scores estimated from subsets of AI task difficulty ratings (for reading, mobility, visual information processing, and visual motor functional domains), and visual ability change scores estimated from difficulty ratings of AI goals (error bars define 95% confidence intervals).

Power analysis demonstrates that with our sample size and a 5% alpha we have 70% power to resolve an average change of 0.065 in the utility index (which corresponds to an effect size of about 0.4). Thus, our sample size is sufficient to conclude that there was no effect of usual low vision care on EQ-5D scores.

DISCUSSION

The results of this study show that the EQ-5D as an outcome measure for low vision patients is severely limited. First, it is heavily skewed toward positive values at baseline, which means the utilities are close to the ceiling with little room to improve for most patients. Yet, these same patients report difficulties with a variety of important daily tasks when measured with the AI. This observation suggests that the items in the EQ-5D are not sensitive to the baseline level of impairment in the low vision population. This observation is consistent with the findings of the comprehensive review by Tosh et al¹⁹ which noted generally poor performance of the EQ-5D especially in patients with macular degeneration and diabetic retinopathy. The EQ-5D has shown variable responsiveness in general studies of visual impairment, such as in cataracts or in distinguishing the impaired population from normals. Second, only four of the five EQ-5D questions potentially can be affected by visual impairment. The remaining domain relates to pain which usually is not caused by diseases that produce visual impairments, nor would one expect pain to be alleviated with vision-specific interventions. The EQ-5D is very broad and vague in its description of symptoms; it does not refer to specific goals and tasks that are important to the patient. In contrast, LVR targets activities that are important to and difficult for the individual patient, likely resulting in effects only for those activities that are targeted by the rehabilitation.

Recommendations from major policy organizations have increased the use of utility estimates and preference-based outcome measures.^4,5,14,20 However, the goal of developing a quality of life outcome measure that is brief, standardized, and allows for cross-disease comparisons limits the sensitivity of the measure. Although low vision can affect the EQ-5D score by limiting mobility and activities of daily living and can contribute to depression and anxiety, LVR aims to improve the patient’s ability to perform specific activities set as rehabilitation goals. LVR (in the United States) rarely directly targets the patient’s psychological state or limitations caused by comorbidities and co-disabilities,²⁹ which are common in the mostly geriatric low vision population. Thus, the sensitivity of the brief and more general self-report measures such as the EQ-5D in LVR is reduced.

This study found that the EQ-5D is unresponsive as an outcome measure for LVR and has poor sensitivity for discriminating low vision patients with different levels of ability. The poor sensitivity to discriminate patients manifests as poor correlations between AI measures of visual or functional ability and EQ-5D utilities. Similar poor correlations were observed in an earlier study that compared EQ-5D utility scores to NEI VFQ-25 scores.¹ As has been suggested by others³⁰, it might be necessary to develop a mapping of visual ability measures onto utilities, for example by having low vision patients who respond to the AI also employ TTO methods to estimate the utility of their health state. Future low vision research must prioritize the development of measures that incorporate patient preferences and can be measured in standard units for policy decision-making and cross-disciplinary comparisons.