Patient-Reported Physical Function Measures in Cancer Clinical Trials

Abstract

Patient-reported outcomes (PROs) are increasingly used to monitor treatment-related symptoms and physical function decrements in cancer clinical trials. As more patients enter survivorship, it is important to capture PRO physical function throughout trials to help restore pretreatment levels of function. We completed a systematic review of PRO physical function measures used in cancer clinical trials and evaluated their psychometric properties on the basis of guidelines from the US Food and Drug Administration. Five databases were searched through October 2015: PubMed/MEDLINE, EMBASE, CINAHL (Cumulative Index of Nursing and Allied Health Literature), Health and Psychosocial Instruments, and Cochrane. From an initial total of 10,233 articles, we identified 108 trials that captured PRO physical function. Within these trials, approximately 67% used the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire and 25% used the Medical Outcomes Study Short Form 36. Both the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire and Medical Outcomes Study Short Form 36 instruments generically satisfy most Food and Drug Administration requirements, although neither sought direct patient input as part of item development. The newer Patient-Reported Outcomes Measurement Information System physical function short form may be a brief, viable alternative. Clinicians should carefully consider the psychometric properties of these measures when incorporating PRO instrumentation into clinical trial design to provide a more comprehensive understanding of patient function.

INTRODUCTION

Symptom monitoring via patient-reported outcome (PRO) measures, defined as any unfiltered report of the status of a patient's health condition that comes directly from the patient (1), is rapidly becoming commonplace in oncology clinical trials (2–5). Real-time capture of treatment-related symptoms and physical function decrements integrates the patient voice into trial conduct and can assist clinicians in understanding the patient's experiences (2–4, 6).

Because of the rapid increase in number of cancer survivors (7), it is especially important to accurately capture PRO physical function (i.e., physical abilities such as walking or reaching that are considered essential for maintaining independence) throughout the conduct of a clinical trial. Availability of such PRO information would allow clinical trialists to monitor potential treatment-related changes from baseline and make recommendations and informed referrals to rehabilitation specialists (e.g., physiatrists, physical therapists) to facilitate the restoration of any losses in physical function as the patient enters survivorship.

A number of well-established and newer measures have been developed to assess PRO physical function in oncology (e.g., European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) (8); Medical Outcomes Study Short Form 36 (SF-36) (9)). In support of this shift toward emphasizing PROs, the National Institutes of Health has completed 2 separate large psychometric initiatives. The first initiative developed a lay-language version of a clinician-based system for documenting and grading adverse events (Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events) (10–12). The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events gives cancer patients the opportunity to provide information about their symptoms/adverse events (e.g., frequency, intensity, and interference with functioning) in addition to the clinician rating. It is important to capture both clinicians’ and patients’ perceptions of adverse events because prior studies have shown a low correlation between the 2 (13, 14).

The second National Institutes of Health initiative involved development and evaluation of a series of item banks to capture nuanced, treatment-related symptom and quality-of-life information (i.e., the Patient-Reported Outcomes Measurement Information System (PROMIS)) (15–18). PROMIS measures are intended to be used across health conditions and thus are not cancer specific like the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events. With numerous choices to include in clinical trial design, it is important to identify all potential PRO physical function measures and compare their psychometric properties in cancer patients to inform clinicians in the selection of tools.

The Food and Drug Administration (FDA) provides regulatory recommendations for the psychometric properties of PRO instruments to be used in drug development, entitled Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (19). This 2009 guidance contains specific criteria for a PRO measure to meet to be used as a clinical outcome assessment for drug development. These criteria include a conceptual framework for the PRO measure, reliability, content validity, construct validity, and ability to detect clinically relevant score changes. Additionally, the FDA has a rigorous formal qualification procedure that reviews the psychometric properties of a candidate PRO measure that is to be potentially used as clinical outcome assessment in a drug development trial relative to the criteria outlined in the FDA PRO guidance (20). Although not all clinical trials in oncology involve medical product development, the psychometric recommendations included in this FDA PRO guidance are reasonable milestones for any measure that is meant to capture this patient information in the clinical trial setting.

The purpose of this article is to systematically identify and review existing PRO measures that have been used to capture physical function in oncology trials. The psychometric properties of these measures will be summarized with respect to criteria outlined in the FDA PRO Guidance, with existing gaps in measurement identified and recommendations made for clinician use in future trials in oncology.

METHODS

Search strategy

A comprehensive electronic literature search for articles was conducted in the following 5 databases: PubMed/MEDLINE (National Library of Medicine); EMBASE (Elsevier); CINAHL (Cumulative Index of Nursing and Allied Health Literature; Elton B. Stephens Co. (EBSCO)); Health and Psychosocial Instruments (OVID; Wolters Kluwer); and CENTRAL (Cochrane Central Register of Controlled Trials; Wiley), through October 2015. No date or language restrictions were applied.

Five main components made up the search strategy used in PubMed/MEDLINE and EMBASE, where a combination of keywords and controlled vocabulary were used (Medical Subject Headings (MeSH) and Emtree, respectively) to describe 1) physical function, 2) cancer, 3) PROs, 4) types of PRO measurement instruments, and 5) properties of PRO measurement instruments.

Synonyms for terms/concepts describing each component part were first searched on individually and combined with the Boolean operator OR. Each individual component search set was then combined with the Boolean operator AND, resulting in a final set of citations that included all of the main component concepts. The CINAHL, Health and Psychosocial Instruments, and CENTRAL searches used only the first 4 components of the strategy: 1) physical function, 2) cancer, 3) PROs, and 4) types of PRO measurement instruments, as the search strategy for these comparatively smaller databases was translated using only keywords. The search terms were as follows: (((instrumentation[sh] OR methods[sh] OR Validation Studies[pt] OR Comparative Study[pt] OR “psychometrics” [MeSH] OR psychometr*[tiab] OR clinimetr*[tw] OR clinometr*[tw] OR “outcome assessment (health care)”[MeSH] OR “outcome assessment”[tiab] OR “outcome measure*”[tw] OR “observer variation”[MeSH] OR “observer variation”[tiab] OR “Health Status Indicators”[MeSH] OR “reproducibility of results”[MeSH] OR reproducib*[tiab] OR “discriminant analysis”[MeSH] OR reliab*[tiab] OR unreliab*[tiab] OR valid*[tiab] OR coefficient[tiab] OR homogeneity[tiab] OR homogeneous[tiab] OR “internal consistency”[tiab] OR (cronbach*[tiab] AND (alpha[tiab] OR alphas[tiab])) OR (item[tiab] AND (correlation*[tiab] OR selection*[tiab] OR reduction*[tiab])) OR agreement[tiab] OR precision[tiab] OR imprecision[tiab] OR “precise values”[tiab] OR test-retest[tiab] OR (test[tiab] AND retest[tiab]) OR (reliab*[tiab] AND (test[tiab] OR retest[tiab])) OR stability[tiab] OR interrater[tiab] OR inter-rater[tiab] OR intrarater[tiab] OR intra-rater[tiab] OR intertester[tiab] OR inter-tester [tiab] OR intratester[tiab] OR intra-tester[tiab] OR interobserver[tiab] OR inter-observer[tiab] OR intraobserver[tiab] OR intraobserver[tiab] OR intertechnician[tiab] OR inter-technician[tiab] OR intratechnician[tiab] OR intra-technician[tiab] OR interexaminer[tiab] OR interexaminer[tiab] OR intraexaminer[tiab] OR intra-examiner[tiab] OR interassay[tiab] OR inter-assay[tiab] OR intraassay[tiab] OR intraassay[tiab] OR interindividual[tiab] OR inter-individual[tiab] OR intraindividual[tiab] OR intra-individual[tiab] OR interparticipant[tiab] OR inter-participant[tiab] OR intraparticipant[tiab] OR intraparticipant[tiab] OR kappa[tiab] OR kappa's[tiab] OR kappas[tiab] OR repeatab*[tiab] OR ((replicab*[tiab] OR repeated[tiab]) AND (measure[tiab] OR measures[tiab] OR findings[tiab] OR result[tiab] OR results[tiab] OR test[tiab] OR tests[tiab])) OR generaliza*[tiab] OR generalisa*[tiab] OR concordance[tiab] OR (intraclass[tiab] AND correlation*[tiab]) OR discriminative[tiab] OR “known group”[tiab] OR factor analysis[tiab] OR factor analyses[tiab] OR dimension*[tiab] OR subscale*[tiab] OR (multitrait[tiab] AND scaling[tiab] AND (analysis[tiab] OR analyses[tiab])) OR item discriminant[tiab] OR interscale correlation*[tiab] OR error[tiab] OR errors[tiab] OR “individual variability”[tiab] OR (variability[tiab] AND (analysis[tiab] OR values[tiab])) OR (uncertainty[tiab] AND (measurement[tiab] OR measuring[tiab])) OR “standard error of measurement”[tiab] OR sensitiv*[tiab] OR responsive*[tiab] OR ((minimal[tiab] OR minimally [tiab] OR clinical[tiab] OR clinically[tiab]) AND (important[tiab] OR significant[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR (small*[tiab] AND (real[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR meaningful change[tiab] OR “ceiling effect”[tiab] OR “floor effect”[tiab] OR “Item response model”[tiab] OR IRT[tiab] OR Rasch[tiab] OR “Differential item functioning”[tiab] OR DIF[tiab] OR “computer adaptive testing”[tiab] OR “item bank”[tiab] OR “cross-cultural equivalence”[tiab]))). For the fifth component (i.e., properties of PRO measurement instruments), search terms from a validated search filter were used (21).

Selection strategy

Studies were deemed eligible for inclusion if they described an oncology clinical trial that included a patient-reported measure of physical function. Both cancer-specific measures and general physical function measures for any health condition were accepted.

Screening process

After removal of duplicates, all titles were randomly assigned to 2 coders (coauthors) and independently reviewed for eligibility. For the abstract reviewing stage, articles were considered if both independent coauthor reviewers reached consensus on eligibility in the prior round. In instances of disagreement, a third coauthor reviewer arbitrated the article. For full-text review, the randomly assigned coders consisted of a primary reviewer and a secondary reviewer for the purposes of verification and quality assurance. Both reviewers independently completed standardized coding forms to extract predetermined information from each potentially eligible article. All reviewers then met as a group and compared full-text article reviews to resolve any potential discrepancies and make final decisions regarding article inclusion. Reference lists from the included full-text articles were also searched to determine whether they should also be considered for inclusion.

Psychometric review

All PRO measures of physical function determined from abstract coding were then reviewed on the basis of the psychometric criteria contained in the FDA PRO Guidance (i.e., conceptual framework, reliability, content validity, construct validity, and ability to detect clinically relevant score changes) (19). Table 1 is a summary of the FDA recommendations.

Table 1.

Summary of FDA Recommendations for the Psychometric Properties of a Patient-Reported Outcomes Instrument

Property	FDA PRO Guidance Recommendations
Conceptual framework	Should be confirmed by using empirical evidence during instrument development
	Explicit statement of relationship among instruments’ concepts, domains, and items
	Response options should be clear and appropriate.
Reliability	Instrument should demonstrate test-retest reliability.
	Instrument should demonstrate internal consistency.
Content validity	Must encompass most important and comprehensive outcomes for patients
	Patient input should be sought for item generation.
	Patient input should be sought until point of saturation.
Construct validity	Obtained results should be consistent with preexisting hypotheses.
	Instrument should have the ability to differentiate between clinically distinct groups.
Clinical relevance of score changes	Instrument should be equally sensitive to gains and losses in health status.
	Instrument should be sensitive to change at all points for the clinical population.

Abbreviations: FDA, Food and Drug Administration; PRO, patient-reported outcome.

RESULTS

The initial electronic literature search yielded a total of 10,233 titles. After duplicates were removed, 8,238 records remained. Following the process of title screening, 2 of the primary authors independently reviewed each of the remaining 1,486 unique article abstracts. Of the 703 articles retained for full-text review, 659 were full-text, 35 were conference proceeding abstracts, and 9 were dissertations. A total of 595 articles were excluded during this phase, the majority of which did not describe findings from clinical trials (n = 462). Additional reasons for article exclusion during full-text review included the following: physical function was not assessed (n = 79), a patient-reported instrument was not used to capture physical function (n = 13), the article described nonoriginal research (n = 9), the study population was noncancer (n = 3), duplicate study (n = 3), the study population comprised adolescents (n = 2), and other (n = 24). A total of 108 articles met eligibility criteria and were included in this review (Figure 1).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow chart.

Characteristics of included studies

Approximately two-thirds of the included studies (67%) utilized the EORTC QLQ-C30 to capture patient-reported physical function (8, 22–87). The EORTC QLQ-C30 is a 30-item PRO measure designed to capture physical, social, emotional, and cognitive well-being; symptoms as they relate to cancer and its treatment; and overall global health and quality of life. The majority (n = 28) of the items are scored by using a 4-point numerical rating scale ranging from 1 (not at all) to 4 (very much). This measure has been validated in over 90 languages and is currently being validated for use in computer-adaptive testing. With respect to the capture of physical function, the first 5 items of the EORTC QLQ-C30 query patients on whether they have had trouble doing strenuous activities, taking a long walk, taking a short walk, whether they need to stay in bed or a chair during the day, and whether they need assistance with daily activities, such as eating or dressing.

Twenty-seven studies (25%) made use of the Medical Outcomes Study Short Form 12 (SF-12) (88, 89), Short Form 20 (SF-20) (90), or SF-36 (91–114). Five included studies made use of both the EORTC QLQ-C30 and SF-36 questionnaires (115–119). The SF-36 is a 36-item measure of general health, mental health, vitality, and pain, as well as physical role functioning, emotional role functioning, social role functioning, and physical function (9). Ten items are specific to physical function and include content related to performing moderate or vigorous activities, lifting or carrying groceries, climbing 1 or several flights of stairs, bending, walking 100 yards (91.4 m) or several hundred yards, walking more than a mile (1.6 km), or bathing or dressing oneself. These items are rated by using a 3-point numerical rating scale to assess functional limitation, ranging from 1 (yes, limited a lot), 2 (yes, limited a little), to 3 (not limited at all). The SF-20 and SF-12 are shorter versions of the SF-36 that include physical function scales with 6 and 2 items, respectively.

Of the remaining 8 (7%) trials, 2 used the Disabilities of the Arm, Shoulder, and Hand (120) Questionnaire (121, 122), 1 made use of The Care Notebook (123, 124), 1 utilized a Linear Analog Scale of Assessment (125), and 4 made use of unidentified or locally developed measures of physical function (126–129).

FDA PRO guidance psychometric review

The EORTC QLQ-C30 and SF-36 were then reviewed by using the FDA PRO Guidance Criteria. Table 2 is a display of the established conceptual framework, reliability, construct validity, content validity, and ability to capture clinically relevant score changes for the EORTC QLQ-C30 and SF-36, respectively. Both the EORTC and SF-36 have established conceptual frameworks. The EORTC QLQ-C30 physical function subscale has reasonable internal consistency (before treatment Cronbach α = 0.68; during treatment Cronbach α = 0.71) and high test-retest reliability (Pearson r = 0.91). The SF-36 physical function subscale has both high internal consistency (Cronbach α = 0.94) and high test-retest reliability at 1-week (Pearson r = 0.74) and 4-week (Pearson r = 0.85) intervals.

Table 2.

Psychometric Review of Patient-Reported Outcomes Physical Function Measures

PRO Physical Function Measure	% Used in Clinical Trial Review	FDA PRO Guidance Recommendations
		Conceptual Framework	Reliability	Content Validity	Construct Validity	Clinical Relevance of Score Changes
EORTC QLQ-C30 (8)	67	5 items representing various aspects of physical function. 4-point NRS indicating degree of impaired physical function.	Internal consistency (before treatment, Cronbach α = 0.68; during treatment, Cronbach α = 0.71); test-retest reliability (Pearson r = 0.91)	No formal patient input for item development; however, patients were debriefed on item comprehension.	Physical function interscale correlation was statistically significant (P < 0.01). No differences in physical function were observed on the basis of disease stage.	Physical function scale was sensitive to change over time on the basis of performance status groups (P < 0.001).
SF-36 (9)	25	10 items representing various aspects of physical function. 3-point NRS indicating level of limitation.	Internal consistency (Cronbach α = 0.94); test-retest reliability (1 week, Pearson r = 0.74; 4 weeks, Pearson r = 0.85)	No formal patient input for item development. Items have been refined through expert consultation.	Physical function correlated 0.90 with physical component of SF-36. Physical function scale successfully discriminates between those who have physical limitations and healthy controls.	Minimally important difference was established by using a distribution-based approach. A 3-point lower physical function T score is associated with being unable to work and a higher 1-year mortality risk.
PROMIS physical function SF (136, 139)	0	10 items measuring physical function of upper extremity, lower extremity, central region, and activities of daily living. 5-point NRS indicating level of physical function.	In cancer patients 6–13 months postdiagnosis, internal consistency was high (Cronbach α = 0.94); test-retest reliability (not determined)	Cognitive interviewing techniques were used to establish patient comprehension.	Physical function SF correlated 0.96 with full PROMIS physical function item bank and 0.88 with the SF-36 physical function scale. The physical function SF discriminates well between those who have high and low physical function and performs well consistently across race/ethnic and age groups.	Minimally important difference was established in advanced-stage cancer patients. Recommended T-score minimally important difference range for 10-item physical functioning is 4.0–6.0 for advanced cancer patients.

Abbreviations: EORTC QLQ-C30, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire; FDA, Food and Drug Administration; NRS, numerical rating scale; PRO, patient-reported outcome; PROMIS, Patient-Reported Outcomes Measurement Information System; SF, short form; SF-36, Medical Outcomes Study Short Form 36.

Both the EORTC QLQ-C30 and SF-36 physical function scales have high interscale correlations, an indicator of construct validity. For content validity, the FDA PRO Guidance recommends the use of cognitive interviewing techniques (130, 131) for item development. The SF-36 and EORTC QLQ-C30 did not include any formal patient input for item development; however, the EORTC QLQ-C30 did include patient debriefing to assess item comprehension.

With respect to clinically relevant score changes, the EORTC QLQ-C30 physical function scale is sensitive to changes over time on the basis of patient performance status. Minimally important differences have been established for the SF-36, with a 3-point reduction in physical function T score associated with being unable to work, as well as a 1-year mortality risk.

DISCUSSION

Given that there are currently over 14.5 million cancer survivors in the United States, with that number expected to triple by the year 2030 (7), it is important to accurately monitor the patient symptomatic experience from the time of trial enrollment and throughout their treatment to provide these individuals with the best care possible and prepare them for survivorship. PROs have been repeatedly shown to be a feasible, acceptable, and reliable source of monitoring this information (4, 132, 133), and there is a high level of acceptability for clinicians to incorporate this patient-reported information into their decision-making (6, 134). Physical function is a quintessential area of focus for patients participating in clinical trials, as many treatments may negatively impact a patient's ability to perform moderate or routine everyday physical activities, ultimately leading to a reduced quality of life. We systematically reviewed clinical trials in oncology to understand the current methods of capturing patient-reported physical function and evaluated the resulting measures relative to established psychometric guidelines (19).

Almost 90% (n = 96) of the reviewed trials included use of the physical function subscales of the EORTC QLQ-C30 or SF-36 to capture PRO information on this domain. In evaluating these subscales relative to the regulatory psychometric recommendations of the FDA PRO Guidance (19), we found that both generically satisfy the minimum requirements for conceptual framework, reliability, construct validity, and establishment of clinically relevant score changes. However, content validity is limited because both the EORTC QLQ-C30 and SF-36 did not directly involve patients in the process of item generation during initial development. Because concepts such as “limitations” and “strenuous” may be highly subjective, particularly in older patients and/or those with more advanced disease, these measures would potentially benefit from the use of qualitative techniques to determine whether the physical function domain items are understood and perceived similarly across patients regardless of demographical or disease type. Both the EORTC QLQ-C30 and SF-36 have been since successfully translated into numerous languages, thus minimizing concern about patient item comprehension.

Approximately 7% of the included trials did not make use of standardized measures that satisfy the FDA PRO Guidance recommendations. While single items or linear analog scale assessment methods may be suitable for the capture of physical function (135), locally developed measures should meet minimum psychometric standards before being included as part of clinical trial conduct.

A shortcoming of both the EORTC QLQ-C30 and SF-36 is that their physical function subscales have not been validated to be administered independently of their core instrument. Although it was not included in any of the reviewed clinical trials, the relatively recently developed Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS-PF) short form (136) may be a viable alternative to monitor this information as part of clinical trial conduct in that it was developed and validated across a wide range of patients with cancer, including varied demographical characteristics (e.g., age, race, ethnicity) and disease type/severity as a brief, stand-alone tool for use across cancer patient subpopulations (17, 137–139). Unlike the EORTC QLQ-C30 and SF-36, the PROMIS-PF did include patient input as part of item development (16, 18, 140), which enhances content validity.

The PROMIS-PF short form (Web Appendix 1, available at http://aje.oxfordjournals.org/) consists of 10 items that assess function in a given patient's lower and upper extremities, as well as their central region in addition to various activities of daily living. These items are scored by using a 5-point numerical rating scale that either asks patients to indicate how limited their activity is (1 = cannot do, 5 = not at all limited) or whether the patient is able to do a given activity (1 = unable to do, 5 = can do without any difficulty). We have included a summary of the psychometric properties of PROMIS-PF relative to the regulatory psychometric recommendations of the FDA PRO Guidance as part of Table 2. PROMIS-PF generically satisfies (17, 137–139) all minimum psychometric criteria of the FDA PRO Guidance, with the exception of not having established test-retest reliability in patients with cancer.

Our findings are potentially affected by our method of defining physical function. In characterizing physical function as physical abilities that are considered essential for maintaining independence, we have subsequently reduced the number of PRO measures that have been established to capture this domain. For example, a number of the reviewed clinical trials incorporated the well-validated and widely used Functional Assessment of Cancer Therapy–General (141) scale to assess “physical well-being.” Although the Functional Assessment of Cancer Therapy–General scale is used to capture health-related quality of life in patients with cancer, its physical well-being domain does not meet our definition of capturing physical function, as its 7 items ask patients about energy level, nausea, pain, meeting the needs of the family, general bother, generally feeling ill, and being forced to spend time in bed. Because these items do not address the core attributes of physical functioning, such as the ability to conduct activities of daily life or limitations on walking, we excluded any trials that exclusively used this tool. An additional limitation may be related to the lack of concordance between patient- and clinician-based reports of physical function (14). Patients of advanced age or with late-stage disease may artificially limit their physical function, or they may have varied subjective definitions of concepts such as “limitations” or “strenuous.”

The present review provides evidence that the vast majority of clinical trials in oncology that have opted to monitor patient-reported physical function have used well-accepted and psychometrically valid methods of capturing this information. When several quality-of-life domains such as emotional, physical, and social functioning are of interest, the EORTC QLC-C30 and SF-36 are reasonable measures because the subscales have been validated as a package. However, if the only PRO domain of interest is physical function, the PROMIS-PF short form may be a viable alternative option that would be of minimal burden to patients. Regardless of which of these 3 measures is used, it is nonetheless essential that clinical trialists incorporate psychometrically sound instrumentation when monitoring PROs to ensure that accurate and reliable patient data are being incorporated into their decision-making process to ultimately improve long-term outcomes for patients.

Abbreviations

EORTC QLQ-C30

European Organization for Research and Treatment of Cancer Quality of Life Questionnaire

FDA

Food and Drug Administration

MeSH

Medical Subject Headings

PRO

patient-reported outcome

PROMIS-PF

Patient-Reported Outcomes Measurement Information System Physical Function

SF-36

Medical Outcomes Study Short Form 36