Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Jan 30.
Published in final edited form as: J Rheumatol. 2017 Mar 15;44(12):1899–1903. doi: 10.3899/jrheum.161112

OMERACT Quality-adjusted Life-years (QALY) Working Group: Do Current QALY Measures Capture What Matters to Patients?

Logan Trenaman 1, Annelies Boonen 1, Francis Guillemin 1, Mickael Hiligsmann 1, Alison Hoens 1, Carlo Marra 1, Will Taylor 1, Jennifer Barton 1, Peter Tugwell 1, George Wells 1, Nick Bansback 1
PMCID: PMC6353610  NIHMSID: NIHMS940543  PMID: 28298567

Abstract

Objective

To understand the limitations with current patient-reported outcome measures (PROM) used to generate quality-adjusted life-years (QALY) in rheumatology, and set a research agenda.

Methods

Two activities were undertaken. The first was a scoping review of published studies that have used PROM to generate QALY in rheumatology between 2011 and 2016. The second was an interactive “eyeball test” exercise at Outcome Measures in Rheumatology 13 that compared sub-domains of widely used generic PROM, as identified through the scoping review, to subdomains of the Assessment of SpondyloArthritis Health Index (ASAS-HI) condition-specific PROM for ankylosing spondylitis.

Results

The scoping review included 39 studies. Five different PROM have been used to generate QALY in rheumatology; however, the EQ-5D and Short Form 6 Dimensions (SF-6D) were used most frequently (in 32 and 9 of included studies, respectively). Special interest group participants identified energy/drive and sleep as 2 key subdomains of the ASAS-HI instrument that may be missed by the EQ-5D, and sexual function as potentially missed by the SF-6D. Participants also expressed concerns that aspects of the process of care and non-health outcomes may be missed. Three ways of incorporating additional subdomains were discussed, including using an alternative generic PROM, modifying an existing generic PROM with “bolt-on” subdomain(s), and generating societal weights for a condition-specific PROM.

Conclusion

Three priorities for future research were identified: understanding whether the EQ-5D and SF-6D identify what matters to patients with different rheumatic conditions, analyzing how much patients value process or non-health outcomes, and identifying which approaches to incorporating a greater number of subdomains into the QALY are being undertaken in other disease areas.

Key Indexing Terms: OMERACT, COST-BENEFIT ANALYSIS, QUALITY-ADJUSTED LIFE-YEARS, RHEUMATOLOGY

Cost-effectiveness analysis is increasingly being used by policymakers to determine which drugs and technologies they will fund. This in turn affects which treatments are available to patients1,2. Quality-adjusted life-years (QALY) are the most widely used measure of benefit to assess the cost-effectiveness of drugs and technologies in healthcare3. QALY have been critical in justifying reimbursement for biologics in many countries4, and will continue to be used while new treatments become available5. The QALY is a measure that considers both quality and length of life6. While length of life is relatively straightforward to measure, measuring “quality of life” is more challenging. In practice, this generally requires 2 components. The first is a system to describe quality of life, and the second is population weights that reflect societal preferences and are specific to that descriptive system.

The descriptive system of health is generally operationalized using a patient-reported outcome measure (PROM) that includes several subdomains that are relevant to health-related quality of life (HRQOL). PROM can be either generic, meaning that they are broadly applicable across different health conditions, or condition-specific, where the subdomains are more focused7. While each has potential advantages and drawbacks, generic PROM are most often used to generate QALY because they are comparable across conditions. The second component required to calculate QALY is population weights, which provide an indication of how much health states are valued. Population weights are generated through a large survey in a representative sample of the general population8, and enable a value judgment to be made about whether, for example, living with reduced mobility is better (or worse) than living in pain. The resulting population weights provide scores for health states, on a 0–1 scale, where 0 corresponds to death and 1 corresponds to perfect health9.

Generally speaking, generic PROM are more widely used as a descriptive system because population weights are available, whereas many condition-specific PROM would require these weights to be generated10. Generating weights can be expensive and difficult, which has led to various “mappings” from condition-specific PROM to generic PROM11, though the accuracy of these algorithms is limited12. Importantly, previous research in rheumatology has shown that using different PROM results in different population weights13,14. National guidelines for economic evaluation recommend the use of generic PROM, such as the EQ-5D, because they are brief and applicable across all conditions15. However, there are concerns that generic PROM may miss aspects of HRQOL that are important to patients16.

The QALY Working Group fits within several core areas of health as defined by the Outcome Measures in Rheumatology (OMERACT), including “Life Impact,” “Death,” and “Resource Use/Economical Impact.”17 The objective of the OMERACT QALY Special Interest Group (SIG) at OMERACT 13 in Whistler, British Columbia, Canada, was to build on previous work18 and analyze the first component of the OMERACT Filter 2.0: Is the instrument a good match with the domain? The specific aim of this SIG was to understand the limitations with current instruments used to generate QALY in rheumatology research, and to set a future research agenda. This was accomplished through 2 activities:

  1. A scoping review to identify which PROM are used to generate QALY in rheumatology, and

  2. An interactive “eyeball test” exercise that compared subdomains of widely used generic PROM to subdomains of the Assessment of SpondyloArthritis Health Index (ASAS-HI) condition-specific PROM for ankylosing spondylitis (AS).

MATERIALS AND METHODS

A scoping review was undertaken to identify which PROM are currently being used to generate QALY in the rheumatology literature. An electronic search of PubMed and Embase was undertaken through OvidSP (Figure 1). To reflect the current literature, only studies published between 2011 and 2016 that used a PROM to generate QALY were included. Studies were excluded if they were dissertations, conference papers or reviews, not published in English, or the full text could not be obtained. Studies were reviewed independently by 2 reviewers (LT, NB) with disagreements resolved through discussion.

Figure 1.

Figure 1

Open in a new tab

Preferred Reporting Items for Systematic reviews and Meta-Analyses diagram. QALY: quality-adjusted life-years; PROM: patient-reported outcome measure; exp: explode subject heading; mp: multipurpose.

To identify subdomains of HRQOL that may be missed by the most widely used PROM, an interactive “eyeball test” was undertaken by attendees of the QALY SIG at OMERACT 13. Participants classified each of the subdomains from the ASAS-HI condition-specific PROM (for AS) as being “directly,” “indirectly,” or “not captured” by the most widely used PROM identified in the scoping review19. The ASAS-HI was chosen because it was developed with patients and considers a broad range of effects, including health, limitations in activities, and social participation. This exercise gave way to an open discussion among participants to identify future research priorities.

RESULTS

The primary literature search identified 776 studies from the online databases, of which 39 studies were included in the final analysis (Figure 1). Across the 39 included studies, 5 preference-based measures were used: the EQ-5D, the Short Form 6 Dimension (SF-6D), the 15D Instrument, the Health Utilities Index Mark 3, and the Quality of Well-Being Scale. The EQ-5D was used most frequently, having been measured in 32 studies across 5 rheumatic conditions. The SF-6D was the second most frequently used measure, with 9 studies measuring across 3 rheumatic conditions.

The 23 participants at the OMERACT SIG included methodologists (n = 8), clinicians (n = 13), and patients (n = 2). In comparing the ASAS-HI to the 2 most widely used PROM from the literature review, the EQ-5D and the SF-6D20,21, participants identified energy/drive and sleep as 2 key subdomains that may be missed by the EQ-5D, and sexual function as potentially missed by the SF-6D (Table 1). While these may be identified indirectly by another subdomain, such as usual activities (EQ-5D), they may not be identified at all22.

Table 1.

The ASAS-HI domains according to the EQ-5D and SF-6D. Participants indicated which domains of the EQ-5D and SF-6D matched the questions of the ASAS-HI and how adequately they covered those questions: directly (++), indirectly (+), not (−), or unclearly (?). The results shown were the most common responses from the participants.

ASAS-HI Domain ASAS-HI Item Description EQ-5D SF-6D
Most Relevant Domain Coverage Most Relevant Domain Coverage
Pain Pain sometimes disrupts my normal activities Pain/discomfort ++ Pain ++
Maintaining a body position I find it hard to stand for long Pain/discomfort + Physical functioning +
Moving around I have problems running Mobility + Physical functioning +
Toileting I have problems using toilet facilities Self-care ++ Social functioning +
Energy and drive I am often exhausted Usual activities ? Vitality ++
Motivation I am less motivated to do anything that requires physical effort Anxiety/depression + Vitality, physical functioning +
Sexual functions I have lost interest in sex Usual activities, anxiety/depression ? Role limitation, mental health ?
Driving I have difficulty operating the pedals in my car Mobility, usual activities + Physical functioning +
Community life I am finding it hard to make contact with people Usual activities + Social functioning ++
Moving around I am not able to walk outdoors on flat ground Mobility ++ Physical functioning ++
Handling stress I find it hard to concentrate Anxiety/depression + Mental health, vitality +
Recreation and leisure I am restricted in traveling because of my mobility Mobility ++ Physical functioning +
Emotional functions I often get frustrated Anxiety/depression ? Mental health +
Washing oneself I find it difficult to wash my hair Self-care ++ Physical functioning +
Economic self-sufficiency I have experienced financial changes because of my rheumatic disease — − − −
Sleep I sleep badly at night Usual activities − Vitality +
Handling stress I cannot overcome my difficulties Anxiety/depression + Mental health +
Open in a new tab

ASAS-HI: Assessment of SpondyloArthritis Health Index; SF-6D: Short Form 6 Dimensions.

In discussing the implications for rheumatological conditions more broadly, participants expressed concerns that the generic PROM focused exclusively on health outcomes, and that patients may also value the process of care and non-health outcomes (such as economic self-sufficiency). It was suggested that in valuing these potential benefits, it would be important to understand whether patients would trade health for them. Additional concerns were raised about whether the wording of levels of the EQ-5D and SF-6D fully identified the context of living with arthritis. On the mobility domain, for example, the levels do not distinguish between walking on flat ground and stairs or hills, and could fail to accurately represent the variability in symptoms on a day-to-day basis. Neither was it clear whether assistive devices, such as specialized footwear or gait aids, should be considered when describing health states.

Three potential ways to incorporate additional subdomains relevant to rheumatology into the QALY were discussed during the OMERACT working group. The first was to use an alternative generic PROM that is not currently being widely used, such as the Computerized Adaptive Tool-5 Domains23. A second approach was to modify an existing measure (e.g., EQ-5D) with “bolt-on” subdomain(s) that are currently missed, which have been done for sleep and vision24,25. The third approach was to generate a set of societal weights for an existing condition-specific PROM. While this can be done through “mapping,” weights from a population survey can provide more accurate estimates26. Working group participants agreed that there is currently insufficient information to recommend one approach over another; however, there was a desire to undertake preliminary work to understand the value and feasibility of these approaches in rheumatology.

DISCUSSION

The QALY working group session at OMERACT 13 analyzed whether the instruments used to generate QALY in rheumatology are a good match with the domain. Through a scoping review and interactive “eyeball test” exercise, participants identified relevant subdomains of the ASAS-HI, which may be missed by the 2 most widely used PROM in rheumatologic trials (EQ-5D and SF-6D).

In the discussion that followed, OMERACT participants also expressed concerns about whether process and non-health outcomes can and should be incorporated into the QALY. There is evidence that rheumatology patients value process and non-health outcomes. For example, patients with rheumatoid arthritis have been shown to value autonomy and participation in shared decision making27, mode of administration28, ongoing disease management29, how informed they are about the treatment29, and their experience of care30, including access to care and attitude of the provider.

Agencies that make reimbursement decisions recommend the use of generic PROM to generate QALY15,31; however, they have also acknowledged in some cases that generic PROM may not be sufficiently sensitive to detect changes in HRQOL. In addition, some policymakers have expressed a desire to consider aspects of convenience of treatment in their decisions32. Thus, efforts are being made across disease areas, such as cancer, to incorporate additional health, non-health, and process subdomains into the QALY33,34. Participants at the QALY working group session expressed a desire for similar activities to be analyzed for rheumatic conditions. Based on the activities and discussion at OMERACT 13, three priorities for research were identified:

  1. To understand to what extent the EQ-5D and SF-6D identify subdomains of HRQOL relevant to patients with different rheumatic conditions, and how their inclusion changes population weights for health states.

  2. To analyze whether aspects of process or non-health outcomes matter to patients, and if so, see whether patients are willing to trade off these potential benefits against health outcomes.

  3. To identify which approaches to incorporating additional (or different) subdomains into the calculation of the QALY are being undertaken in different disease areas.

References

  • 1.Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. JAMA. 1996;276:1172–7. [PubMed] [Google Scholar]
  • 2.Drummond MF, Sculpher MJ, Torrance G, O’Brien B, Stoddart G. Methods for the economic evaluation of health care programmes. third. Oxford: Oxford University Press; 2005. [Google Scholar]
  • 3.Mehrez A, Gafni A. Quality-adjusted life years, utility theory, and healthy-years equivalents. Med Decis Making. 1989;9:142–9. doi: 10.1177/0272989X8900900209. [DOI] [PubMed] [Google Scholar]
  • 4.Brennan A, Bansback N, Reynolds A, Conway P. Modelling the cost-effectiveness of etanercept in adults with rheumatoid arthritis in the UK. Rheumatol. 2004;43:62–72. doi: 10.1093/rheumatology/keg451. [DOI] [PubMed] [Google Scholar]
  • 5.Latest recommendations from NICE: some positive, et al…. PharmacoEcon Outcomes News. 2016;744:35. [Google Scholar]
  • 6.Weinstein MC, Torrance G, McGuire A. QALYs: the basics. Value Health. 2009;12(Suppl 1):S5–9. doi: 10.1111/j.1524-4733.2009.00515.x. [DOI] [PubMed] [Google Scholar]
  • 7.Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care. 1989;27(Suppl):S217–32. doi: 10.1097/00005650-198903001-00018. [DOI] [PubMed] [Google Scholar]
  • 8.Torrance GW, Feeny D. Utilities and quality-adjusted life years. Int J Technol Assess Health Care. 1989;5:559–75. doi: 10.1017/s0266462300008461. [DOI] [PubMed] [Google Scholar]
  • 9.Gold MR, Stevenson D, Fryback DG. HALYs and QALYs and DALYs, oh my: similarities and differences in summary measures of population health. Annu Rev Public Health. 2002;23:115–34. doi: 10.1146/annurev.publhealth.23.100901.140513. [DOI] [PubMed] [Google Scholar]
  • 10.Neumann PJ, Goldie SJ, Weinstein MC. Preference-based measures in economic evaluation in health care. Annu Rev Public Health. 2000;21:587–611. doi: 10.1146/annurev.publhealth.21.1.587. [DOI] [PubMed] [Google Scholar]
  • 11.Bansback N, Marra C, Tsuchiya A, Anis A, Guh D, Hammond T, et al. Using the health assessment questionnaire to estimate preference-based single indices in patients with rheumatoid arthritis. Arthritis Rheum. 2007;57:963–71. doi: 10.1002/art.22885. [DOI] [PubMed] [Google Scholar]
  • 12.Longworth L, Rowen D. Mapping to obtain EQ-5D utility values for use in NICE health technology assessments. Value Health. 2013;16:202–10. doi: 10.1016/j.jval.2012.10.010. [DOI] [PubMed] [Google Scholar]
  • 13.Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ. 2004;13:873–84. doi: 10.1002/hec.866. [DOI] [PubMed] [Google Scholar]
  • 14.Kwakkenbos L, Fransen J, Vonk MC, Becker ES, Jeurissen M, van den Hoogen FH, et al. A comparison of the measurement properties and estimation of minimal important differences of the EQ-5D and SF-6D utility measures in patients with systemic sclerosis. Clin Exp Rheumatol. 2013;31(Suppl 76):50–6. [PubMed] [Google Scholar]
  • 15.National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. [Internet. Accessed January 30, 2017.] Available from: www.nice.org.uk/article/pmg9/chapter/Foreword. [PubMed]
  • 16.Versteegh MM, Leunis A, Uyl-de Groot CA, Stolk EA. Condition-specific preference-based measures: benefit or burden? Value Health. 2012;15:504–13. doi: 10.1016/j.jval.2011.12.003. [DOI] [PubMed] [Google Scholar]
  • 17.Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d’Agostino MA, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67:745–53. doi: 10.1016/j.jclinepi.2013.11.013. [DOI] [PubMed] [Google Scholar]
  • 18.Boonen A, Maetzel A, Drummond M, Suarez-Almazor M, Harrison M, Welch V, et al. The OMERACT Initiative. Towards a reference approach to derive QALY for economic evaluations in rheumatology. J Rheumatol. 2009;36:2045–9. doi: 10.3899/jrheum.090355. [DOI] [PubMed] [Google Scholar]
  • 19.Kiltz U, van der Heijde D, Boonen A, Cieza A, Stucki G, Khan MA, et al. Development of a health index in patients with ankylosing spondylitis (ASAS HI): final result of a global initiative based on the ICF guided by ASAS. Ann Rheum Dis. 2015;74:830–5. doi: 10.1136/annrheumdis-2013-203967. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Tijhuis GJ, de Jong Z, Zwinderman AH, Zuijderduin WM, Jansen LM, Hazes JM, et al. The validity of the Rheumatoid Arthritis Quality of Life (RAQoL) questionnaire. Rheumatology. 2001;40:1112–9. doi: 10.1093/rheumatology/40.10.1112. [DOI] [PubMed] [Google Scholar]
  • 21.Rudwaleit M, Landewé R, van der Heijde D, Listing J, Brandt J, Braun J, et al. The development of Assessment of SpondyloArthritis international Society classification criteria for axial spondyloarthritis (part I): classification of paper patients by expert opinion including uncertainty appraisal. Ann Rheum Dis. 2009;68:770–6. doi: 10.1136/ard.2009.108217. [DOI] [PubMed] [Google Scholar]
  • 22.Lillegraven S, Kvien TK. Measuring disability and quality of life in established rheumatoid arthritis. Best Pract Res Clin Rheumatol. 2007;21:827–40. doi: 10.1016/j.berh.2007.05.004. [DOI] [PubMed] [Google Scholar]
  • 23.Kopec JA, Badii M, McKenna M, Lima VD, Sayre EC, Dvorak M. Computerized adaptive testing in back pain: validation of the CAT-5D-QOL. Spine. 2008;33:1384–90. doi: 10.1097/BRS.0b013e3181732a3b. [DOI] [PubMed] [Google Scholar]
  • 24.Yang Y, Brazier J, Tsuchiya A. Effect of adding a sleep dimension to the EQ-5D descriptive system: a “bolt-on” experiment. Med Decis Making. 2014;34:42–53. doi: 10.1177/0272989X13480428. [DOI] [PubMed] [Google Scholar]
  • 25.Longworth L, Yang Y, Young T, Mulhern B, Hernández Alava M, Mukuria C, et al. Use of generic and condition-specific measures of health-related quality of life in NICE decision-making: a systematic review, statistical modelling and survey. Health Technol Assess. 2014;18:1–224. doi: 10.3310/hta18090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Brazier JE, Rowen D, Mavranezouli I, Tsuchiya A, Young T, Yang Y, et al. Developing and testing methods for deriving preference-based measures of health from condition-specific measures (and other patient-based measures of outcome) Health Technol Assess. 2012;16:1–114. doi: 10.3310/hta16320. [DOI] [PubMed] [Google Scholar]
  • 27.Nota I, Drossaert CH, Taal E, Vonkeman HE, van de Laar MA. Patient participation in decisions about disease modifying anti-rheumatic drugs: a cross-sectional survey. BMC Musculoskelet Disord. 2014;15:333. doi: 10.1186/1471-2474-15-333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Bansback N, Trenaman L, Harrison M. How important is mode of administration in treatments for rheumatic diseases and related conditions? Curr Rheumatol Rep. 2015;17:514. doi: 10.1007/s11926-015-0514-3. [DOI] [PubMed] [Google Scholar]
  • 29.de Wit MP, Smolen JS, Gossec L, van der Heijde DM. Treating rheumatoid arthritis to target: the patient version of the international recommendations. Ann Rheum Dis. 2011;70:891–5. doi: 10.1136/ard.2010.146662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.El Miedany Y, El Gaafary M, Youssef S, Ahmed I, Palmer D. The arthritic patients’ perspective of measuring treatment efficacy: Patient Reported Experience Measures (PREMs) as a quality tool. Clin Exp Rheumatol. 2014;32:547–52. [PubMed] [Google Scholar]
  • 31.Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. (3rd) 2006 [Internet. Accessed February 16, 2017] Available from: www.cadth.ca/media/pdf/186_EconomicGuidelines_e.pdf.
  • 32.National Institute for Health and Care Excellence. Diagnostics assessment programme. [Internet. Accessed January 30, 2017.] Available from: www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-diagnostics-guidance. [PubMed]
  • 33.Brennan VK, Dixon S. Incorporating process utility into quality adjusted life years: a systematic review of empirical studies. Pharmacoeconomics. 2013;31:677–91. doi: 10.1007/s40273-013-0066-1. [DOI] [PubMed] [Google Scholar]
  • 34.Higgins A, Barnett J, Meads C, Singh J, Longworth L. Does convenience matter in health care delivery? A systematic review of convenience-based aspects of process utility. Value Health. 2014;17:877–87. doi: 10.1016/j.jval.2014.08.2670. [DOI] [PubMed] [Google Scholar]

RESOURCES