Some Comments on Mapping from Disease-Specific to Generic Health-Related Quality-of-Life Scales

Abstract

An article by Lu et al. in this issue of Value in Health addresses the mapping of treatment or group differences in disease-specific measures (DSMs) of health-related quality of life onto differences in generic health-related quality-of-life scores, with special emphasis on how the mapping is affected by the reliability of the DSM. In the proposed mapping, a factor analytic model defines a conversion factor between the scores as the ratio of factor loadings. Hence, the mapping applies to convert true underlying scales and has desirable properties facilitating the alignment of instruments and understanding their relationship in a coherent manner. It is important to note, however, that when DSM means or differences in mean DSMs are estimated, their mapping is still of a measurement error–prone predictor, and the correct conversion coefficient is the true mapping multiplied by the reliability of the DSM in the relevant sample. In addition, the proposed strategy for estimating the factor analytic mapping in practice requires assumptions that may not hold. We discuss these assumptions and how they may be the reason we obtain disparate estimates of the mapping factor in an application of the proposed methods to groups of patients.

Introduction

Measuring health-related quality of life (HRQOL) has become standard in clinical trials of treatments and in studies assessing the impact of health conditions. There is little agreement, however, on the choice of instrument to achieve this. It is often felt that disease-specific measures (DSMs) of HRQOL are most sensitive to changes in HRQOL related to a specific condition or disease. For example, the Minnesota Living with Heart Failure (MLHF) questionnaire [1,2] may be chosen to assess the progress of patients with congestive heart failure entering intensive treatment, the 25-item National Eye Institute Visual Function Questionnaire (NEI-VFQ-25) [3] to assess the benefit of eye surgery, or the Beck Depression Inventory [4] to measure the impact of a treatment for depression. However, it is desirable to express treatment impact on HRQOL in terms that allow comparison to other health conditions, and preference scoring for application in cost-effectiveness analysis. When only a DSM is available from a trial, there may be a need to convert (map or cross-walk) the scores of the MLHF questionnaire to the EuroQol five-dimensional (EQ-5D) questionnaire or other generic HRQOL utility measure. In this issue of the journal, Lu et al. [5] address methods for mapping DSMs into generic scores.

The word “mapping” has taken on several meanings, at least two of which are relevant to Lu et al. [5] and this discussion. The first is anticipated in the Introduction of Lu et al. [5], in which “mapping” is defined as “treatment effects estimated on the disease-specific measure (DSM) are multiplied by a mapping coefficient to convert them into treatment effect estimates on the generic QOL scale.” What the authors end up developing, however, is a second conceptual mapping applicable to the true, not estimated, mean score or treatment effect. This second factor loading–based mapping informs about the relationships between what the instruments truly attempt to measure. Because it pertains to a theoretical framework not mudded by measurement error in a particular setting, it has more desirable properties. As noted in Lu et al. [5], the framework has been used in educational testing to “align instruments,” that is, to ascertain that they fundamentally measure the same underlying abilities. One needs to be clear about the distinction and how it affects the correct way to perform a conversion in practice. In this commentary, I attempt to clarify how the proposed methods may be put to use and also further discuss the assumptions needed for the specific methods proposed by Lu et al. [5].

Two different approaches to estimating the mapping between true underlying means are proposed. (It should be noted that models in Lu et al. [5] are actually not presented in terms of differences, but in terms of group means, later to be differenced.) Both approaches depend on the availability of data or results from a separate study that measured the DSM and obtained simultaneous scores on at least one generic HRQOL instrument. Briefly, the first method fits a linear regression of the generic score on the DSM and then applies corrections for measurement error through knowing the reliability of the DSM. The second method assumes that there are data on two generic scores together with the DSM and that the two generic scores share the aspect—and only the aspect—of health covered by the DSM. Linear correlations are then used to extract a conversion factor. We illustrate the impact of the assumptions by applying the methods proposed in Lu et al. [5] to a study of the MLHF questionnaire in congestive heart failure clinics and the NEI-VFQ-25 in patients undergoing cataract extraction [6]. This study also obtained simultaneous data on several generic instruments, as well as repeated measures, allowing the extraction of reliability coefficients for the MLHF questionnaire and the NEI-VFQ-25.

Comment on Methods

The Role of Reliability in Mapping

The measurement error, reliability, and underlying factors of HRQOL instruments have received relatively little attention in the cost-effectiveness analysis and HRQOL literature, with some exceptions [e.g., 7–10], and Lu et al. [5] should be lauded for bringing renewed attention to these important measurement aspects. As pointed out, in an ordinary regression analysis of a generic HRQOL instrument on a DSM, the regression coefficient is affected by the reliability of the DSM in the particular sample. Specifically, the slope of the relationship is b_x = βρ_x, where β is the regression coefficient that would be obtained had the true value of DSM been known and ρ_x is the reliability of the predictor X (i.e., DSM), obtainable, for example, from test/retest data. Nonetheless, the equation using b_x as the mapping slope is correct (unbiased) for the particular study when X is either a measured individual or an estimated mean score.

It is relevant to note that ρ_x is the ratio ${\sigma }_{\text{a}}^2/({\sigma }_{\text{a}}^2+{\sigma }_{\text{e}}^2),$ where ${\sigma }_{\text{a}}^2$ is the variance between individuals in the true x (DSM) and ${\sigma }_{\text{e}}^2$ is the measurement error variance. Then, the estimated coefficient for obtaining a generic score from a DSM may vary between studies with different underlying ${\sigma }_{\text{a}}^2$ (i.e., spread in DSM), and formulating the mapping as one between true values (i.e., in terms of β) is advantageous. However, when DSM values measured in practice are to be mapped, β still needs to be multiplied by a ρ_x appropriate to the sample under consideration. In fact, it may be desirable to build into a study the estimation of ρ_x, for example, via a test-retest component. When taking the mean of DSM across a treatment group, both variance components become averaged so that the reliability coefficient is

$$\left(\frac{{\sigma }_{\text{a}}^2}{\text{n}}\right)/(\frac{{\sigma }_{\text{a}}^2}{\text{n}}+\frac{{\sigma }_{\text{e}}^2}{\text{n}})=\frac{{\sigma }_{\text{a}}^2}{({\sigma }_{\text{a}}^2+{\sigma }_{\text{e}}^2)}$$

which is identical to that of the initial measurement. Hence, b_x = βρ_x should be used in the mapping of estimated means, albeit ideally with a ρ_x modified to fit the specific study.

Role of Linearity of the Relationship

It should be noted that the estimation of the relationship between the generic score and the DSM via b_x = βρ_x depends on the regression of the observed generic score on DSM being linear. Given the strong ceiling effects in many generic scores, this assumption may not hold, and b_x across the range of X may depend on the HRQOL distribution of the sample. Several approaches to the linearity/ceiling effect problem have been applied and investigated in the literature, such as using splines [11], censored or latent class models [12], discretizing the scores [10], or fitting different models to different ranges of the DSM [5]. The mappings proposed [5] would need further work as to how reliability would be estimated and applied in such models. It should be noted that this problem is separate from the linearity assumed in the factor analytic relationship, which is between underlying unobserved constructs.

Assumptions in Using Two Generic Scores

Another proposed way to obtain the measurement error– corrected mapping of the DSM onto a generic instrument requires that the DSM be available in an auxiliary study together with two generic instruments [5]. A rather strict assumption is made that the components in the two instruments not captured by the DSM are uncorrelated. In other words, the component(s) the two generic instruments have in common are those and only those captured by the DSM. One may envision that such an assumption can be true if the DSM is comprehensive enough to capture all standard domains of HRQOL. On the other hand, even if the generic instruments capture all domains of the DSM affected by the disease under study, it is possible that individuals have other health condition, or even unexpected side effects of treatment captured by both generic instruments, but not by the DSM. In the latter case, the assumption would not hold. It has been shown that many generic instruments have much variance in common [9,10], requiring careful consideration before the assumption of nonshared extraneous-to-DSM domains can be made.

Equation 10 in Lu et al. [5] provides the estimator. For consistency, we use X to denote values of the DSM, Q1 to denote values of the generic instrument desired, and Q2 of the auxiliary generic instrument:

$$\beta =(\text{Sign of relationship})\left(\right|\frac{\mathit{\text{corr}}(\text{Q}1,\text{Q}2)}{\mathit{\text{corr}}(\text{Q}1,\text{X})}\left|\right)\sqrt{\left(\frac{\mathit{\text{Var}}\left(\text{Q}1\right)}{\mathit{\text{Var}}\left(x\right)}\right)}$$

In this formula, the expression under the square-root is simply a conversion of the two instruments to the same scale, and the sign converts, for example, a DSM scale where lower value indicates better HRQOL to a generic scale where higher value indicates better HRQOL. The ratio of correlations, however, will capture the needed quantity only if corr(Q1,Q2) is not inflated by shared extraneous-to-DSM factors. The feasibility of the assumption likely depends on the level of health in the sample used, as generic instruments have been shown to capture underlying dimensions differentially at different levels of health [9]. Ceiling effects in Q1 can reduce the linear correlation between Y and Q1, possibly more so than the correlation between Q1 and Q2, where ceiling may be shared. It is difficult to assess the impact of the linearity assumption between Q1 and X, except to note that nonlinearity–based bias in the directly reliability adjusted and the two-instrument–based estimators will be in opposite directions, as the linear relationship is in the numerator of the former and in the denominator of the latter.

Results of Application

Problems arising with the assumptions discussed above can be summarized as 1) a linear relationship between the measured generic score and the DSM may not hold because of ceiling effects in most generic scores and 2) generic scores may share health dimensions not measured by the DSM, except for specific DSMs and generic scores. We now turn to examining the performance of the methods in a patient sample evaluated with a number of HRQOL measures [6].

The Clinical Outcomes and Measurement of Health Study [6] provides longitudinal data on the NEI-VFQ-25 [3] in a group of patients undergoing cataract surgery and on the MLHF questionnaire [1,2] for a group entering clinics to manage congestive heart failure. As individuals were measured at three time points (baseline, 1 month, and 6 months), we estimate reliability from the last two visits, and map the NEI-VFQ-25 and the MLHF questionnaire, respectively, onto the EQ-5D questionnaire at 1 month. Our reliability of the NEI-VFQ-25 was estimated at 0.72 and that of the MLHF questionnaire at 0.75, the former within published range for individual scales of 0.68 to 0.91 and the latter somewhat lower than published 0.86 [2,3]. As the study also administered the short form 36 health survey version 2, six-dimensional health state short form (derived from short form 36 health survey), health utilities index mark 3, and the Quality of Well Being Self-Administered (QWB-SA) scale [13–16], we use each of these as a secondary instrument. At face value, both the NEI-VFQ-25 and the MLHF questionnaire share considerable content with the generic instruments.

Results of applying the methods proposed in Lu et al. [5] to individuals who provided data on all instruments at 1-month follow-up are shown in Table 1, together with partial correlations of generic instruments with the EQ-5D questionnaire (all P < 0.05). Note that because the intention is to apply the conversion to treatment differences, intercepts are not needed. We see that for the NEI-VFQ-25 the method based on two generic instruments tends to yield considerably higher coefficients than the reliability-based method, while the results are more consistent for the MLHF questionnaire. We note the following: 1) The EQ-5D questionnaire has a strong ceiling effect in the cataract surgery sample (38% have value = 1) and a moderate ceiling in the congestive heart failure sample (19%). The reliability-based NEI-VFQ-25 conversion is likely underestimated, at least in the cataract surgery sample. 2) The correlation of the EQ-5D questionnaire with NEI-VFQ-25 is only 0.41, while the correlation with the MLHF questionnaire is 0.70; this may have affected the stability of the mapping in the cataract sample. 3) Only health utilities index mark 3 shares with the EQ-5D questionnaire a sharply skewed distribution in these samples. However, the results using health utilities index mark 3 do not differ from those using other instruments with no notable ceiling, perhaps indicating that joint content is the most important issue. 4) The mental short form 36 health survey component also utilized by Lu et al. [5] has the lowest partial correlation. 5) We have previously shown that especially the QWB-SA shares considerable content with the Minnesota Living With Heart Failure Questionnaire (LHFQ) [17]. Despite this, the partial correlation with the EQ-5D questionnaire is nonnegligible.

Table 1.

Estimated mapping coefficients of two disease-specific measures into the EQ-5D questionnaire.

	Reliability adjusted	By SF36P	By SF36M	By SF-6D	By HUI3	By QWB-SA
The MLHF questionnaire to the EQ-5D questionnaire (n = 77)	−0.0069	−0.0072	−0.0060	−0.0083	−0.0077	−0.0078
Partial on the MLHF questionnaire correlation with the EQ-5D questionnaire		0.44	0.24	0.54	0.54	0.46
NEI-VFQ-25 to the EQ-5Dquestionnaire (n = 191)	0.0092	0.021	0.016	0.024	0.024	0.023
Partial on NEI-VFQ-25 correlation with the EQ-5D questionnaire		0.53	0.31	0.52	0.52	0.49

EQ-5D, EuroQol five-dimensional; HUI3, health utilities index mark 3; MLHF, Minnesota Living with Heart Failure; NEI-VFQ-25, the 25-item National Eye Institute Visual Function Questionnaire; SF36P, short form 36 physical component; SF36M, short form 36 mental component; SF-6D, six-dimensional health state short form (derived from short form 36 health survey); QWB-SA, Quality of Well Being Self-Administered.

Future Directions

More attention needs to be paid to improving the reliability of HRQOL measures, which is generally quite low [8]. Outcome studies should incorporate test-retest features into their design to directly assess reliability in their population. Furthermore, much more attention to the underlying structures of DSMs versus generic measures is needed. Our application together with that in Lu et al. [5] illustrates the difficulties of knowing whether assumptions hold. Possibly the mappings envisioned by Lu et al. [5] would be more fruitfully applied directly in a latent variable framework than estimated by making the present assumptions. The fact that DSMs are needed at all is thought provoking, and more research is needed into the overlap or lack there of with generic scales.

Finally, a method of regression analysis needs to be agreed upon that accommodates ceiling effects. It appears that simply allowing censoring at the maximum value is not sufficient [12].