A new health-related quality of life instrument for leukemia: will it be widely adopted soon?

Introduction

The rapid escalation in the treatment costs of cancer, as well as modest survival gains of the treatment, drive an emerging need to determine the value of cancer therapy from the perspectives of stakeholders such as patients, payers, and health care administrators. Despite the fact that global health-related quality of life (HRQoL) is not considered as the primary efficacy endpoint for cancer therapy approval by the Food and Drug Administration (FDA) [1], there is an increasing role of HRQoL evidence in guiding a clinician’s treatment decision and influencing a payer’s coverage decision. In the field of leukemia research, there has been a paucity of validated leukemia-specific measurement tools to assess HRQoL [2], and consequently limited patient-reported outcome (PRO) studies involving leukemia patients [3]. News that might be of interest to researchers developing novel HRQoL tools or leukemia treatment therapies is that the void for a leukemia-specific HRQoL instrument has recently been filled by FACT-Leu [4]. This instrument is a new pharmacoeconomics tool that will help address part of the question of the value of leukemia therapy. The FACT-Leu questionnaire was developed largely in conjuction with the Functional Assessment of Cancer Therapy-General (FACT-G), a general instrument with 27 items. Items are typically specific questions or descriptions associated with each dimension of the overall functional assessment of the PRO instrument [5]. For example, the items associated with the measurement of emotional wellbeing could include ‘frustration with activity limitation’, ‘discouraged by illness’, ‘worry about illness’, and ‘emotional ups and downs’. FACT-Leu retained a total of 17 items specific to leukemia after a process of validation. As a disease specific instrument with a relatively short list of questions, FACT-Leu will significantly reduce the administration time on the part of patients and clinicians. Will this new instrument be widely accepted and easily adopted in different settings? In this paper, we attempt to shed some light on this specific question by providing an overview of several strengths and weaknesses of the development process of the instrument.

Strengths

The major strengths of the study [4] include the use of cross-culturally relevant items, a list of comprehensive items with face and content validity, improved reliability compared to other instruments, and convenience in application as explained below.

A collection of cross-culturally relevant items

The majority of HRQoL measurements tools for cancer have been developed in Europe [6,7]. Recognizing the need for quality-of-life assessment tools with international applicability [8], the developers of FACT-Leu set out to create a culturally relevant HRQoL tool for acute and chronic leukemia. Item generation included inputs from participants from South America, Europe, and North America, but demographic data were not available. Hoping to answer the need for leukemia-specific HRQoL measurements tools, the authors supplemented the widely used 27-item FACT-G with a 17-item leukemia sub-scale to create FACT-Leu.

Comprehensive items with face and content validity

To ensure an exhaustive list of items, researchers interviewed patients and medical experts, and completed detailed literature research to generate a comprehensive list of items. FACT-Leu included the concerns of leukemia patients that were not addressed in the Medical Research Council/European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Leukemia scale (MRC/EORTC QLQ-Leu), such as infertility and sexual functioning [7]. Face and content validity in PRO instruments refers to the extent to which an instrument measures the important aspects of concepts that developers or users intend to assess [9]. Patients reported FACT-Leu as being “relevant” and “comprehensive”, demonstrating that face and content validity had been established from the patients’ point of view.

Improved reliability compared to the other instruments

FACT-Leu exhibited a higher degree of internal consistency compared to its earlier predecessor MRC/EORTC QLQ-Leu. Hair and colleagues defined reliability as an assessment of the degree of consistency between multiple measurements of a variable [10]. This study assessed the consistency of the entire questionnaire with Cronbach’s alpha each time the questionnaire was administered. Alpha coefficients range in value from 0 to 1 and are used as indices to describe the reliability of questionnaires/scales. The higher the score, the more reliable the generated scale. All values yielded alpha coefficients ranging from 0.86 to 0.88, and exceeded the values of 0.70 suggested as an acceptable reliability coefficient by Hair and colleagues [10]. QLQ-Leu offers a significant improvement in reliability compared to the MRC/EORTC -Leu for which Cronbach’s alpha is lower, ranging from 0.58 to 0.79 [7]. The administration of FACT-Leu at baseline, and then again 3–7 days later, showed strong test–retest stability with an intraclass correlation coefficient of 0.861. Yet, the selected time frame of 3–7 days is relatively short and memory bias cannot be excluded.

Convenient to use and easy to understand by patients

Participants reported the FACT-Leu scale as being “easy to understand”. Using a Likert scale, participants were instructed to select their responses based on how they felt in the past 7 days. Minimal participant training allowed for the scale to be completed within 5–15 minutes.

Weaknesses

Despite the strengths listed above, one must also pay attention to several limitations of the method used to develop FACT-Leu.

Weak validity evaluation

The study lacks sound data to support convergent validity. In HRQoL research, convergent validity generally refers to whether the HRQoL scale relates (converges) to other measures of related constructs, which are typically quantitatively assessed using Spearman and Pearson correlations [10]. The study examined convergent validity by comparing FACT-Leu with MRC/EORTC QLQ-Leu, which is the only existing leukemia-specific HRQoL questionnaire but an instrument for long term sequelae of leukaemia treatments [2,4]. The magnitude of Spearman correlation coefficients ranged from 0.29 to 0.63; however, researchers did not comment on the overall strength of correlation between the two scales, which left readers wondering if FACT-Leu compares favorably to the other leukemia-specific questionnaire. The low correlation between the two instruments might be due to the fact that MRC/EORTC QLQ-Leu, an item module scale, is for assessing long-term effects of treatment in leukemia patients who have been in complete remission for 1–2 years, as opposed to the short follow-up period in this study [7,11]. It is also worth noting that MRC/EORTC QLQ-Leu has not been validated and the EORTC does not recognize it as an official formal scale for leukemia [12]. Understandably, it is a challenge for this study to provide sound validity testing due to the lack of a gold-standard instrument in the area of leukemia therapy.

Another issue related to validity is the lack of external construct validity evaluation, in which the hypothetical relationship between relevant clinical parameters and scale scores is supposed to be tested. Oncologists choose outcomes such as survival time, time to disease progression and tumor response as objective data to evaluate cancer treatments. Integration of HRQoL along with biomedical endpoints has been emphasized recently in clinical trials [1]. However, HRQoL has been accepted by the FDA in the USA and the European Medicines Agency as one of the methods by which new drug labeling approvals may be obtained [5,13]. Hence, the instrument could have been more valuable in clinical trials if it had shown that HRQoL outcomes are aligned with other clinical measurements or more objective measures.

And finally, the validation was conducted in a limited sample. Although FACT-Leu included international input during scale construction, scale validation was limited to patients from three oncology clinics in Chicago, Illinois. The race or ethnicity of the validation sample was predominantly White (91%). Inclusion criteria also required participants to be minimally sufficient in English to be able to complete interviews and forms, thus limiting the scale’s international applicability.

Lack of rigorous sensitivity testing data over time

Sensitivity or responsiveness of an instrument refers to the extent to which it can detect the changes in measured concepts over time. Sensitivity is a necessary property for a rigorous PRO instrument along with reliabity and validity [5]. If patients are expected to experience change in a measured concept of quality of life due to a response to the treatment, then the values for the PRO instrument measuring that concept should change. In other words, lack of change in the PRO score when there is clear evidence demonstrating a patient’s experience in the concept, indicates that the instrument has inadequent sensitivity. The study did show that patients with improved performance status also showed improved leukemia subscale scores. However, there was insufficient sample size in each subgroup experiencing varying levels of performance status change and no mention of the time interval between different measurements. Given that FACT-Leu is an instrument developed for both acute and chronic leukemia, one should expect the study to test the sensitivity in different time frames.

Small sample in the phase of scale construction and a low proportion of treated patients in validation sample

Despite the fact that no consensus exists regarding how to determine an appropriate sample size for scale construction or validation [14], saturation is a typical technique recommended by ISPOR PRO Good Research Practices Task Force. In the data collection process, any point beyond the point of saturation will elicit no new relevant information [14]. In this study, scale construction included the input of 29 patients and 16 health care providers, which was determined by saturation. However, one would expect a larger sample size for scale construction for a heterogeneous multi-cultural sample, which typically requires a larger sample size [14]. On the other hand, a required sample size of 50 was determined for validation in this study through power analysis and the researchers managed to recruit a sample of 79 participants. It is also noted that although scale validation began with around 76 participants at baseline (in the original study some samples have 76 subjects and some 77), only 61 participants completed the final set of questionnaires. However, an explanation for the loss of participants was not provided in the study [4].

The researchers of this study did not find any significant differences in FACT-Leu scores between acute leukemia patients (n=34) and chronic leukemia patients (n=44). Further validation studies using a larger and more diverse patient population of acute and chronic leukemia patients is required before suggesting that this tool may be applicable to acute lymphoblastic leukemia, chronic myelogenous leukemia, and chronic lymphocytic leukemia patients. In addition, the validation sample is composed of only 38% of patients treated by chemotherapy. As a result, insufficient data from those treated patients might compromise the rigor of the findings and limit the potential for use in evaluating cancer therapy. It follows that this instrument’s validity might be unclear when the cancer therapy to be evaluated has many new side effects or adverse events reported. This explains why patients felt the scale would have been more relevant at diagnosis or during treatment. In contrast to the MRC/EORTC QLQ-Leu scale, which best assesses patients in remission, items describing symptoms such as fevers, chills, loss of appetite, and infections are perhaps more directly related to treatment rather than post-treatment or during remission.

Conclusions

FACT-Leu is a relatively new HRQoL measurement tool that attempts to fill the gap for leukemia-specific HRQoL measurement tools. The scale has already been implemented in an American-based clinical trial assessing the use of a tyrosine kinase inhibitor and HRQoL in chronic myeloid patients [3,15]. Results from this study showed high questionnaire completion rates. Trask and colleagues obtained baseline FACT-Leu scores that were comparable to scores obtained from the validation study [15]. This provides evidence of reproducibility and reliability through “inter-rater” testing. Due to the lack of validated scales available for researchers to use [2] and the emerging need to understand HRQoL in leukemia patients, it is very likely that FACT-Leu will be readily accepted by medical and research professionals. An increasing number of clinical trials in cancer research have been incorporating HRQoL as part of their trial designs, either as a primary or secondary endpoint, over the last decade [16]. However, caution should be exercised when considering this HRQoL instrument for use in registered trials. Although this tool fills a void for an HRQoL instrument for leukemia, its future value in informing clinical decision-making will be limited unless it can help clinicians and patients to easily interpret HRQoL outcomes and distinguish implications of different numerical scales. FACT-Leu shows great promise in the advancement of PRO tools but additional validation data are required before the tool can be deemed a valid and internationally applicable scale. Fortunately, guidelines for efforts toward developing new PRO instruments have recently started to emerge. The FDA created the Study Endpoints and Label Development (SEALD) group to devise new methodologies for PRO instrument development to facilitate decisions related to the approval of drugs, labels, and promotional claims based on PROs [17]. The guideline and regulations provided by them should point future research in PRO development in the right direction.

Abbreviations

EORTC

European Organization for Research and Treatment of Cancer

FACT-G

Functional Assessment of Cancer Therapy-General

FACT-Leu

The Functional Assessment of Cancer Therapy – Leukemia

FDA

US Food and Drug Administration

HRQoL

health-related quality of life

MRC

Medical Research Council

PRO

patient-reported outcome

QLQ-Leu

Quality of life Questionnaire-Leukemia scale

SEALD

Study Endpoints and Label Development