Abstract
Objectives
There is a need for a brief symptom index for advanced kidney cancer that includes perspectives of both patients and clinicians and is consistent with the Food and Drug Administration’s guidance for patient-reported outcome measures. This study developed and examined the preliminary reliability and validity of the new National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy (FACT)-Kidney Symptom Index 19.
Methods
Fifty patients with advanced kidney cancer provided open-ended and survey responses ranking their most important symptoms. Responses were reconciled with published clinician reports of the most important symptoms. Ten experienced oncologists rated symptoms as disease- or treatment-related. Patients completed quality-of-life and performance status measures.
Results
A 19-item index was produced from symptoms that were rated as most important by patients or clinicians. It includes three subscales: disease-related symptoms (DRS), treatment side effects (TSE), and general function and well-being (FWB). Internal consistency was good for the full instrument (α = 0.83), the DRS subscale (α = 0.76), and the FWB subscale (α = 0.78) but lower for the TSE subscale (α = 0.59). Convergent validity was demonstrated through correlations with the FACT-General. Patients with differing performance status were distinguished by the total score (F2,47 = 17.37; P < .0001), the DRS subscale (F2,47 = 14.22; P < .0001), and the FWB subscale (F2,47 = 13.40; P < .0001) but not the TSE subscale (F2,47 =1.48; P = 0.2380).
Conclusions
The National Comprehensive Cancer Network/FACT-Kidney Symptom Index 19 combines symptoms deemed most important by patients and clinicians. Preliminary evidence suggests that the total score and DRS and FWB subscales are reliable and valid as summary indexes. The TSE subscale may be least relevant given the advent of newer therapies.
Keywords: cancer, patient-reported outcomes, quality of life, symptoms
Introduction
Every year in the United States, there are 64,770 new cases of kidney cancer, and an estimated 13,570 deaths [1]. Almost twice as many men than women are affected, with it representing about 5% of all new cancer diagnoses in men [1]. Approximately 25% to 30% of individuals with kidney cancer are diagnosed with metastatic disease [2]. For those treated for local disease, about 25% recur, typically with distant metastases [3]. Surgery is the primary treatment for early stage disease. Relative 5-year survival rates are significantly worse for distant (11%) and regional (63%) disease when compared with local disease (91%) [1]. Prior to 2005, the only available systemic treatment options for metastatic disease were cytokines, such as interleukin-2 and interferon-alpha. Since that time, several molecular-targeted therapies have been approved by the Food and Drug Administration (FDA) for metastatic kidney cancer, including sorafenib (Nexavar), sunitinib (Sutent), temsirolimus (Torisel), everolimus (Afinitor), bevacizumab (Avastin), pazopanib (Votrient), and axitinib (Inlyta) [4]. Newer targeted therapies offer promise for improved clinical outcomes [5]; however, cure remains an elusive goal [6].
Evaluating the clinical benefit of kidney cancer treatment must include an appreciation for health-related quality of life (HRQOL), including symptoms of disease and their impact on functioning and life enjoyment. Systemic therapies used for advanced disease contribute to declines in HRQOL. As patients are presented with second-and third-line treatment options, HRQOL associated with these options can be a useful guide when making treatment decisions [7]. Several HRQOL instruments have been used in kidney cancer research, including the EuroQoL five-dimensional questionnaire[8], the Medical Outcomes Study short-form 36 health survey [9], the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire C30 [10], the Renal Cell Carcinoma Symptom Index [11], and the Functional Assessment of Cancer Therapy (FACT)-Kidney Symptom Index (FKSI) [12]. The FKSI was developed by using both patient and clinician input about the most important symptoms to assess during drug therapy for advanced kidney cancer. The 15-item instrument has demonstrated good internal consistency, test-retest reliability, convergent and discriminant validity, and responsiveness to change in clinical status [12]. It has been used as the primary patient-reported outcome measure for kidney cancer trials [13,14]. To reduce patient burden, a 10-item reliable and valid version of the FKSI was used in trials [12,14]. In an attempt to isolate disease-related symptoms, a 9-item subset of disease-related symptoms (DRS) was drawn from the FKSI-15 by a panel of clinical experts [15]. This DRS, also with good reliability and validity [12], was included as an outcome in clinical trials (e.g., [13,16]).
The iterative development of these brief, symptom-focused indexes occurred prior to the FDA’s Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims [17]. Consequently, they do not explicitly ensure content validity in a manner consistent with the Guidance. For example, although the FKSI-15 development did use patient input for item generation and review for clarity and readability, this does not strictly meet the recommendation of the FDA Guidance, which identifies the centrality of patient involvement in final selection and content through qualitative research methodology. The Guidance also emphasizes reaching saturation in item generation. A more explicit open-ended questioning, prior to presenting a list of previously identifed items, might help ensure that an instrument would meet this requirement.
The National Comprehensive Cancer Network (NCCN) embarked on an effort to develop and validate symptom indexes for advanced cancer treatment by using a methodology [7] that includes both patient and expert clinician input regarding relevant symptoms with an emphasis on the patient’s perspective. Consequently, we developed the NCCN/FACT-Kidney Symptom Index (NFKSI) for advanced kidney cancer in accordance with the FDA Guidance, and examined its preliminary reliability and validity.
Methods
The present study was part of a larger study that sought to develop 11 symptom indexes for different types of advanced cancer by using a multistep process [7]. First, patients with advanced kidney cancer answered open-ended questions to identify their highest priority cancer symptoms. Next, patients’ responses were combined with results from previous surveys of oncology clinicians to create a symptom index including the most important patient- and oncology clinician-rated symptoms specific to advanced kidney cancer. Next, expert oncologists rated the identified symptoms as predominantly disease- or treatment-related. This was used to create subscales within the measure. Finally, initial validation analyses were conducted on data collected from patients.
Patient Participants
Adult patients with stage III or IV kidney cancer receiving care at one of four NCCN member institutions or a community cancer support organization in 2005 and 2006 were eligible for participation. Participants had received at least two cycles of chemotherapy or 1 month of noncyclical chemotherapy to ensure that patients would have some experience with treatment-related symptoms and had no other primary malignancy (excluding nonmelanoma skin cancer) in the previous 5 years. NCCN accrual sites included Duke Comprehensive Cancer Center in Durham, NC; Robert H. Lurie Comprehensive Cancer Center of Northwestern University in Chicago, IL; H. Lee Moffitt Cancer Center & Research Institution in Tampa, FL; Fred Hutchinson Cancer Research Center/Seattle Cancer Care Alliance in Seattle, WA; and NorthShore University Health System in Evanston, IL, which at the time was affiliated with the Robert H. Lurie Comprehensive Cancer Center of Northwestern University. A private, nonprofit social service organization, Cancer Wellness Center of Northbrook, IL, also served as an accrual site. All participants as judged by their treating physician and study staff had sufficient cognitive ability to provide informed consent and were fluent in reading and speaking English. All participants provided informed consent in accordance with institutional review board and Health Insurance Portability and Accountability Act (HIPAA) guidelines prior to participating and completed all measures at one time point. Participants received $50 in exchange for their participation.
Patient Procedures
After providing basic demographic information, patients completed a two-part survey. First, patients were asked to “Please think of the full range of your experience receiving drug treatment for your illness. Please tell me what you think are the most important symptoms or concerns to monitor when assessing the value of drug treatment for your illness.” This open-ended prompt was intended to identify concepts not currently included in the Kidney Cancer Symptom/Concern Checklist (Checklist). Patients were asked to rate each of the identified symptoms on a scale from 0 (not important to you) to 10 (extremely important to you). Second, patients were provided the Checklist and asked to identify the 10 most important symptoms and write in additional symptoms. Then, patients were asked to reduce their list of top 10 symptoms down to the 5 “very most important” symptoms. The Kidney Cancer Symptom/Concern Checklist [12,18] consists of 26 symptoms identified by expert clinicians to be related to cancer in general and specific to kidney cancer. To control for potential response bias due to item order, four versions of the Checklist presenting symptoms in different order were administered.
Patients next completed the FACT-General (FACT-G) [19] and the FKSI-15 [12] with duplicate items administered only once. The FACT-G total and subscales (Physical Well-being [PWB], Functional Well-being [FWB], Emotional Well-being [EWB], and Social Well-being [SWB]) were used in analyses. Higher scores on all FACT instruments indicate better QOL. Patients completed the Eastern Cooperative Oncology Group Performance Status (ECOG-PS) [20], which assesses performance as fully ambulatory without symptoms (0), fully ambulatory with some symptoms (1), requiring less than 50% of awake time to rest (2), requiring more than 50% of awake time to rest (3), or bedridden (4).
Analysis of Patient Data
Frequency distributions were analyzed from the patient-generated symptom list. Symptoms/concerns that were identified by at least 10% of the sample were retained for further consideration. If a treatment-related symptom was identified by 20% or more of the patients, it was retained in the final instrument. Identified symptoms were analyzed to determine whether they were already represented on the Checklist. If not, they were reviewed further to select an item from the Functional Assessment of Chronic Illness Therapy (FACIT) measurement system to include in the final instrument. FACIT items were used where the patient-generated content matched an existing item because they had already been tested for comprehension, validity, and translatability.
Frequency distributions of patients’ five most important Checklist symptoms/concerns were created. A “write-in” symptom was retained for review if at least three (5%) patients included it. The frequency of chance endorsement of a symptom was calculated by dividing 5 (the allowable number of “very most important symptoms”) by 26, the number of items in the Checklist, and multiplying this by 50 (number of participants). This established a cutoff threshold. Items that met or exceeded this cutoff were retained for consideration in the final instrument. Items that were within a point of this cutoff were retained to err on the side of inclusion. Symptoms that were endorsed at rates well above the cutoff threshold were evaluated to determine whether an additional symptom from the same domain merited inclusion.
Archival Data Review
Previously in the development of the FKSI-15 [12], kidney cancer clinicians rated the most important symptoms to include in a kidney cancer symptom index. These published rankings were reviewed by the current study team. Symptoms that had been identified in clinicians’ top 10 were retained for possible inclusion in the new instrument.
Physician Participants and Procedures
Ten oncology physicians were recruited via e-mail with assistance of the NCCN central office. All treating physicians at all 21 NCCN member institutions were potentially eligible to participate. In the case of kidney cancer, there was typically only one physician at each member institution who met eligibility criteria (at least 3 years’ experience treating at least 100 patients with advanced kidney cancer). All physicians provided informed consent in accordance with institutional review board and HIPAA guidelines and received $100 for their participation.
Physicians completed the Checklist augmented by additional symptoms (51 total) in an online survey [21]. Physicians rated each symptom/concern on a five-point scale: “exclusively disease-related,” “predominantly disease-related,” “too close to determine” (or neither disease-nor treatment-related), “predominantly treatment-related,” and “exclusively treatment-related.”
Analysis of Physician Data
Clinicians’ ratings of symptom cause were collapsed into three categories: exclusively/predominantly disease-related, too close to determine/neither, and exclusively/predominantly treatment-related. Items rated in the first category by more than half of the experts were labeled “disease-related symptoms” (DRS); those rated by more than half of the experts as treatment-related were labeled “treatment side effects” (TSE). Remaining items were not symptoms and were therefore classified as “function/well-being” (FWB), reflecting their underlying general content.
Scale Construction
The NFKSI-19 includes symptoms/concerns that were 1) spontaneously generated from patients in open-ended questions, 2) ranked by patients on the Checklist as most important, or 3) ranked by clinicians in previous research as being most important to assess. Items from the FACIT measurement system were selected to represent each symptom/concern. Higher scores on the NFKSI-19 indicate greater symptom burden. Items were then assigned to a subscale (TSE, DRS, FWB) on the basis of expert allocation.
Preliminary Reliability and Validity Analyses
Preliminary analyses on NFKSI-19’s reliability and validity were conducted by analyzing data from items also included in the FACT-G and FKSI-15 collected for this study. Cronbach’s alpha assessed the internal consistency reliability of NFKSI-19 total and subscales. Convergent validity was assessed by examining Spearman correlations among the NFKSI-19 total and subscales and the FACT-G total and subscale scores after removing overlapping items that would inflate the correlation. One-way analyses of variance examined the relationship between ECOG-PS and NFKSI-19 to assess criterion validity.
Results
Participant Characteristics
Fifty patients (mean age = 59.4 ± 10.5 years; range 33–79 years) participated. Participants were predominantly male (64.0%), White (94.0%), highly educated (66.0% with greater than high school), and not working outside of the home (72.0%). Most patients (78.0%) reported currently experiencing symptoms or requiring daytime rest. Table 1 presents the sociodemographic characteristics of the participants.
Table 1.
Sociodemographic characteristics of study participants.
| Characteristics | % (N = 50) |
|---|---|
| Mean age ± SD (y) | 59.4 ± 10.5 |
| Gender | |
| Female | 36 |
| Male | 64 |
| Race/ethnicity | |
| African American | 6 |
| White | 94 |
| Non-Hispanic | 96 |
| Highest education | |
| Some high school or less | 6 |
| High school graduate or GED | 28 |
| Vocational college or some college | 22 |
| College degree | 26 |
| Professional or graduate degree | 18 |
| Current occupational status | |
| Homemaker | 2 |
| Unemployed | 2 |
| Retired | 34 |
| On disability | 24 |
| On leave of absence | 10 |
| Part-time employed | 8 |
| Full-time employed | 20 |
GED, general educational development.
The physicians had a mean age of 49.3 ± 5.7 years, and all but one were male. Years of experience treating advanced kidney cancer ranged from 6 to more than 20. Sixty percent had treated more than 500 patients with advanced kidney cancer. The 10 physicians were from nine different institutions.
Symptom Identification
Fourteen symptoms were generated by at least 10% of the patients in response to an open-ended prompt (mean = 4.5 ± 3.7 symptoms generated per participant; see Table 2). Most common were fatigue (48%), concern about treatment effectiveness (36%), and nausea (31%). The symptoms identified as most important by 42 patients (eight patients had missing data in this phase) were concern about treatment effectiveness (mean = 9.3), being able to function normally (mean = 8.4), and weakness (mean = 8.0). In an attempt to be inclusive, three concerns (finances, blood pressure, and sleep issues) identified just below the cutoff were retained for further review. Diarrhea was identified as a new concept not included on the Checklist and was within a percentage point of the a priori cutoff. Given its increasing importance in the era of targeted chemotherapy, it was retained. Other new concepts did not meet the cutoff for retention of treatment-related symptoms (e.g., hair loss, skin issues, and taste issues), were considered outside the scope of the HRQOL concept (e.g., financial status or concerns), or were considered to be a sign (a manifestation of a condition observable by others such as blood pressure), not a symptom. This resulted in the retention of 11 symptoms for the final instrument.
Table 2.
Participant-generated symptoms and their perceived importance.
| Symptom | Rank | % of responses (n = 42) | Mean importance rating (0–10) | Symptom/concern in checklist |
|---|---|---|---|---|
| Fatigue | 1 | 48 | 6.2 | Yes |
| Treatment effectiveness | 2 | 36 | 9.3 | Yes |
| Nausea | 3 | 31 | 6.5 | Yes |
| Side effects | 4 | 24 | 6.5 | Yes |
| Diarrhea | 5 | 19 | 7.0 | No |
| Weight loss | 6 | 17 | 7.1 | Yes |
| Fever | 6 | 17 | 7.0 | Yes |
| Hair loss | 6 | 17 | 5.6 | No |
| Appetite | 6 | 17 | 5.9 | Yes |
| Skin issues | 10 | 14 | 6.5 | No |
| Taste issues | 10 | 14 | 5.3 | No |
| Function normally | 12 | 12 | 8.4 | Yes |
| Pain | 12 | 12 | 7.0 | Yes |
| Weakness | 14 | 10 | 8.0 | No |
| Finances | 15 | 7 | 9.0 | No |
| Blood pressure | 15 | 7 | 8.7 | No |
| Sleep issues | 15 | 7 | 7.3 | Yes |
Seven of the 26 symptoms on the Checklist were endorsed by patients as the most important to monitor at a probability greater than chance (see Table 3). Two additional items were endorsed within one point of the cutoff. All these concepts were retained for the final measure. Being able to sleep well was ranked by patients just below the a priori threshold. Given its mention on the free response portion of the survey as well as its ranking on the Checklist, it was retained in the final measure. Other symptoms with equal Checklist ratings included nausea (already included in the final measure) and “Being able to enjoy the things I usually do for fun.” Because the latter was not mentioned in the free response results and its concept was covered by the higher ranked item “Being able to enjoy life,” it was removed from further consideration. No write-in symptoms met the threshold for retention; most were cited only by one patient.
Table 3.
Comparison of patient and oncology clinician symptom rankings.
| Symptom | Patient overall rank (% endorsed) | Oncology clinician overall rank* |
|---|---|---|
| Side effects of treatment | 1 (52) | 2 |
| Lack of energy (fatigue) | 2 (42) | 2 |
| Fatigue | 3 (34) | – |
| Worry that my condition will get worse | 4 (30) | – |
| Being able to enjoy life | 5 (28) | – |
| Being able to work (include work at home) | 6 (20) | – |
| Pain | 6 (20) | 4 |
| I am content with the quality of my life right now | 8 (18) | – |
| Shortness of breath | 8 (18) | 7 |
| Nausea | 10 (16) | 13 |
| Being able to enjoy the things I usually do for fun | 10 (16) | 20 |
| Being able to sleep well | 10 (16) | 25 |
| Appetite | 13 (14) | 5 |
| I have bone pain | 14 (10) | 7 |
| I am losing weight | 17 (8) | 1 |
| I am bothered by fevers | – | 5 |
| Because of my physical condition, having trouble meeting the needs of my family | 17 (8) | 7 |
| I am losing hope in the fight against my illness | 17 (8) | 10 |
| Forced to spend time in bed | 20 (6) | 10 |
| I have been coughing | 20 (6) | 10 |
Data from Cella et al. [12].
Among patients’ top priority symptoms, only four (side effects of treatment, lack of energy, pain, and shortness of breath) were rated by clinicians as important at a probability greater than chance in the FKSI-15 development study (see Table 3) [12]. Three of the clinician top-priority items were not retained in the final measure. “Losing hope in the fight against my illness” was considered to be the same concept as the higher ranked “Worry that my condition will get worse.” Because it was not rated highly by patients and was not one of the top five clinician-rated symptoms, it was omitted. Having trouble meeting the needs of family because of one’s physical condition and being forced to spend time in bed were eliminated on the basis of low patient and expert rankings, and lack of specificity to kidney cancer.
Item Selection for the NFKSI-19
FACIT items were identified for the selected patient-generated and Checklist domains. One rare symptom (“I have had blood in my urine”) had been included in the FKSI-15 and was retained because of its importance clinically. This resulted in the inclusion of 19 items in the NFKSI-19: a 13-item DRS subscale, a 3-item TSE subscale, and a 3-item FWB subscale (see Table 4).
Table 4.
NFKSI-19 items by subscale across kidney cancer symptom indices.
| Subscale | Item | NFKSI-19 | FKSI-15 | FKSI-10 | FKSI-DRS | FACT-G |
|---|---|---|---|---|---|---|
| DRS-P | I have a lack of energy | ✓ | ✓ | ✓ | ✓ | ✓ |
| I have pain | ✓ | ✓ | ✓ | ✓ | ✓ | |
| I am losing weight | ✓ | ✓ | ✓ | ✓ | ||
| I feel fatigued | ✓ | ✓ | ✓ | ✓ | ||
| I have been short of breath | ✓ | ✓ | ✓ | ✓ | ||
| I am bothered by fevers (episodes of high body temperature) | ✓ | ✓ | ✓ | |||
| I have bone pain | ✓ | ✓ | ✓ | ✓ | ||
| I have been coughing | ✓ | ✓ | ✓ | |||
| I feel weak all over | ✓ | |||||
| I have had blood in my urine | ✓ | ✓ | ✓ | |||
| I have a good appetite | ✓ | ✓ | ✓ | |||
| I am sleeping well | ✓ | ✓ | ✓ | |||
| DRS-E | I worry that my condition will get worse | ✓ | ✓ | ✓ | ✓ | |
| TSE | I have nausea | ✓ | ✓ | |||
| I have diarrhea (diarrhoea) | ✓ | |||||
| I am bothered by side effects of treatment | ✓ | ✓ | ✓ | ✓ | ||
| FWB | I am able to work (include work at home) | ✓ | ✓ | ✓ | ||
| I am able to enjoy life | ✓ | ✓ | ✓ | ✓ | ||
| I am content with the quality of my life right now | ✓ | ✓ |
DRS-E, Disease-Related Symptoms-Emotional; DRS-P, Disease-Related Symptoms-Physical; FACT-G, Functional Assessment of Cancer Therapy-General; FKSI, Functional Assessment of Cancer Therapy-Kidney Symptom Index; FKSI, Functional Assessment of Cancer Therapy-Kidney Symptom Index; FWB, General Function and Well-Being; NFKSI-19, National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy-Kidney Symptom Index 19; TSE, Treatment Side Effects.
Clinician Ratings Symptom Attribution
Two symptoms (bother by treatment side effects and nausea) were categorized by clinicians as TSE. Diarrhea was added for reasons mentioned above [22–24]. Clinicians’ ratings of the remaining 16 symptoms were reviewed by the senior author (D. C.) for scale assignment. Three items (able to enjoy life, able to work, and content with quality of life) were categorized as FWB. One item (worry that condition will get worse) was considered an emotional DRS, thus generating a distinction between DRS-P (physical) and DRS-E (emotional).
Preliminary Reliability and Validity
Two of the final NFKSI-19 items were not in the original FACT-G or FKSI-15 (“I feel weak all over” and “I have diarrhea”; see Table 4). Therefore, these items were treated as missing, and scores on the NFSKI-19 were prorated according to the FACIT scoring protocol [25]. Internal consistency reliability for the NFKSI-19 was good (17 items with data, α = 0.83). While the DRS subscale (12 items with data, α = 0.76) and the FWB subscale (α = 0.78) demonstrated adequate international consistency reliability, the TSE subscale (2 items with data, α = 0.59) displayed less adequate internal consistency reliability.
Associations between the NFKSI-19 and the FACT-G total and subscales were evaluated with Spearman correlation coefficients corrected for overlapping items (see Table 5). The total NFKSI-19 was moderately to strongly correlated with the FACT subscale and total scores apart from the FACT SWB subscale. The NFKSI-19 DRS showed a similar pattern of correlations. The NFKSI-19 TSE was correlated only with the PWB subscale (r = 0.61, P < .0001). The NFKSI-19 FWB was moderately to strongly correlated with the FACT-G, FACT PWB subscale, and FACT FWB subscale (though this was expected as all three NFKSI-19 FWB items are from the FWB subscale).
Table 5.
Spearman correlation coefficients of NFKSI-19 with FACT-G subscales, corrected for overlapping items where applicable
| Instrument/subscale | PWB | SWB | EWB | FWB | FACT-G |
|---|---|---|---|---|---|
| NFKSI-19 | 0.76 P < 0.0001 |
0.24 P = 0.0889 |
0.41 P = 0.0031 |
0.67 P < 0.0001 |
0.68 P < 0.0001 |
| NFKSI-19 DRS | 0.75 P < 0.0001 |
0.26 P = 0.0633 |
0.42 P = 0.0026 |
0.56 P < 0.0001 |
0.68 P < 0.0001 |
| NFKSI-19 TSE | 0.61 P < 0.0001 |
0.09 P = 0.5197 |
0.26 P = 0.0716 |
0.09 P = 0.5410 |
0.36 P = 0.0098 |
| NFKSI-19 FWB | 0.52 P < 0.0001 |
0.25 P = 0.0790 |
0.33 P = 0.0198 |
0.78 P < 0.0001 |
0.70 P < 0.0001 |
DRS, Disease-Related Symptoms; EWB, Emotional Well-being; FACT-G, Functional Assessment of Cancer Therapy-General; FWB, Function/Well-Being; NFKSI-19, National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy-Kidney Symptom Index 19; PWB, Physical Well-being; SWB, Social Well-being; TSE, Treatment Side-Effects.
The ability of the NFKSI-19 to discriminate between patients’ performance status (ECOG-PS) was analyzed. Because only four patients reported having a performance status of 3, these patients were collapsed with category 2. Scores on the NFKSI-19 differed by ECOG-PS in the expected direction (see Table 6), with higher scores on the NFKSI-19 corresponding to better performance status. The total NFKSI-19 (F2,47 = 17.37,; P < .0001), DRS subscale (F2,47= 14.22; P < .0001), and FWB subscale (F2,47 = 13.40; P < .0001) all significantly discriminated between the three categories of performance status. The TSE subscale did not discriminate performance status scores (F2,47 = 1.48; P = 0.2380).
Table 6.
NFKSI-19 score means and SDs by performance status.
| ECOG-PS | N | NFKSI-19 | DRS | TSE | FWB |
|---|---|---|---|---|---|
| 0 (no symptoms) | 11 | 57.0 ± 9.1 | 38.4 ± 6.7 | 8.9 ± 3.5 | 9.6 ± 2.2 |
| 1 (ambulatory with symptoms) | 22 | 53.4 ± 7.3 | 37.0 ± 4.8 | 8.6 ± 3.2 | 7.9 ± 1.8 |
| 2 (<50% rest) or 3 (>50% rest) | 17 | 39.2 ± 10.4 | 27.4 ± 7.8 | 7.1 ± 2.3 | 5.1 ± 3.1 |
DRS, Disease-Related Symptoms; ECOG-PS, Eastern Cooperative Oncology Group Performance Status; FWB, General Function and Well-Being; NFKSI-19, National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy-Kidney Symptom Index 19; TSE, Treatment Side Effects.
Discussion
The NFKSI-19, a new symptom index that includes symptoms rated as highest priority from patients with advanced kidney cancer and experienced clinicians, was developed by using methods consistent with the FDA’s Guidance [17]. Qualitative data on symptoms most important to patients were consistent with previous research, suggesting that fatigue, TSE, and worry about one’s condition worsening are primary concerns. NFKSI-19 items offer support for the content validity of previous instruments (e.g., FKSI-15 and FKSI-DRS). The content of the FKSI-15 was supplemented with additional symptoms of weakness, nausea, diarrhea, and contentment with QOL.
It was notable that there was limited overlap between patient and clinician views on the most important symptoms. Clinicians rated weight loss as most important [12], yet it ranked 17th among patients. This discrepancy is consistent with previous work [26–30]. Clinicians may more readily focus on symptoms that suggest disease progression or factors that may delay or reduce treatment (e.g., bone pain and fever), whereas patients may more readily identify psychosocial symptoms (e.g., worry and ability to work) as most relevant. In addition, relatively rare symptoms (e.g., weight loss) that tend to occur late in the disease course will not be rated as important by patients who have not experienced it or do not realize that it is an ominous harbinger. Yet, it is a very important clinical symptom that warrants inclusion in a priority index for advanced disease. Our experience in scale construction and these findings in particular demonstrate the benefit of including both patient and clinician perspectives to create a fuller understanding of HRQOL in advanced kidney cancer and provide a more adequate capture of the perspective of a patient with advanced kidney cancer.
Clinicians’ assessments on the cause of symptoms were consistent with previous work [15]; NFKSI-19 items were able to be sorted into a 13-item DRS subscale, a 3-item TSE subscale, and a 3-item FWB subscale. As the NFKSI-19 includes all items from the FKSI-15, FKSI-10, and FKSI-DRS, it is possible to use the NFKSI-19 and calculate scores for these earlier measures to facilitate analyses with previously collected data. Because of the inclusion of four items on the NFKSI-19 that were not originally part of the FKSI-15, it is not possible to directly calculate NFKSI-19 scores from the FKSI-15. However, one can prorate scores from FKSI data to derive scores comparable to the NFKSI-19 if more than 50% of the 19 NFKSI items are completed: 19 × [(sum of NFKSI item responses)/(number of NFKSI items completed)] [25]. The NFKSI-19 DRS, TSE, and FWB subscales offer advantages over other HRQOL instruments; clearly designated subscales separate clusters of symptoms that may be of greater interest to patients or clinicians or more relevant for a specific clinical research protocol. This is particularly relevant in kidney cancer considering that seven drugs have been approved by the FDA for treating metastatic disease since 2007. Whereas it is unlikely that the symptoms associated with underlying malignancy or its progression are going to change, the toxicity profile of current treatments can change rapidly. Hence, future work should develop strategies to more frequently update treatment-related symptom scales. NFKSI-19 scores can therefore be reported by subscale.
The NFKSI-19 demonstrates preliminary reliability and validity. The total measure, DRS, and FWB subscales demonstrate good internal consistency. Only two (of the three) items in the TSE subscale were tested in this study (I have nausea; I am bothered by side effects of treatment); the low alpha (0.59) is not surprising given one item taps a specific side effect, while the other asks broadly about side effects. Therefore, any side effects other than nausea will contribute to responses on one but not the other question, introducing error. Convergent validity was demonstrated through correlations with the FACT-G total and sub-scales. The NFSKI-19 is focused on physical and functional symptoms, and so poor correlations with social well-being were expected. The NFKSI-19 also demonstrates criterion validity in distinguishing patients with differing performance status.
Given the overlap among these measures, guidelines for instrument selection may be useful. We recommend the NFKSI-19 when the goal is focused assessment of symptoms in advanced kidney cancer. Among the FKSI options (see Table 4), it best ensures content validity from the perspective of the FDA Guidance [17]. All other FKSI options are embedded within the NFKSI-19, and could therefore be scored from the administration of the NFKSI-19. This was demonstrated by Butt et al. [31] who calculated NFKSI-19 total, DRS, FKSI-15, and FKSI-DRS-9 norms for the general U.S. adult population. If a briefer assessment is needed, any of the subordinate versions (FKSI-15, FKSI-10, FKSI-DRS) are valid options.
Implications
As the measurement of clinically meaningful patient outcomes in advanced cancer has evolved, a greater emphasis has been placed on HRQOL and patient-reported disease-specific symptoms. As a result, the development of measures of disease-specific symptoms has become essential to the comprehensive assessment of HRQOL in clinical trials and practice, particularly with respect to advanced cancers for which curative options are limited. The present study extends previous findings in kidney cancer through the development and initial validation of a symptom measure specific to advanced kidney cancer. Improved clinical utility in the measurement of disease-specific HRQOL has the potential to improve symptom monitoring and intervention, patient satisfaction with treatment, medical decision making, and health policy. It also provides a foundation for better assessment of the HRQOL effects of disease versus emerging treatments in kidney cancer. The NFKSI-19 is the most comprehensive and up-to-date kidney cancer–specific instrument available that was developed by using methods consistent with the FDA’s Guidance [17].
Limitations
The study sample was predominantly White and male and included patients with only advanced kidney cancer. Thus, generalizability of findings to earlier stage disease and other sociodemographic groups is limited. The same 50 patients identified NFKSI-19 content and provided initial validation data. Because of the cross-sectional study design, it was not possible to assess test-retest reliability or sensitivity to change. Future research focused on longitudinal data collection in a more diverse sample is needed for further validation. Moreover, in contrast to the DRS and FWB subscales, the TSE subscale demonstrated notably less adequate reliability and validity, reflecting an important area for further research. Finally, it is likely that because of treatment advances since this data collection, there is a different profile of high-priority current treatment-related symptoms. Advantages of the NFKSI-19 are that 1) the treatment-related symptoms are identified in their own subscale allowing for the instrument adaptation over time without affecting the other subscales and 2) the inclusion of the generic item “I am bothered by side effects of treatment” enables patients to respond on the basis of their own experience of side effects. Regardless, it is still likely worthwhile to retest reliability and validity including an assessment of content validity for any patient-reported outcome instrument periodically to ensure that the instrument is still reflecting current priority symptoms because available treatment and supportive care choices may change. The frequency of retesting should reflect the velocity of change in the treatment landscape.
Source of financial support: Support for the study was provided by grants from the following pharmaceutical companies: Amgen, AstraZeneca, Bayer, Bristol-Myers Squibb, Centocor, Cell Therapeutics, Inc., Genentech, GlaxoSmithKline, Eli Lilly and Company, Merck & Co., Novartis, Ortho Biotech, Pfizer, Sanofi-Aventis, and Takeda Pharmaceuticals. The contents represent original work. Dr. Abernethy has research funding from the US National Institutes of Health, US Agency for Healthcare Research and Quality, Robert Wood Johnson Foundation, Biovex, DARA, Helsinn, MiCo, and Pfizer; these funds are all distributed to Duke University Medical Center to support research including salary support for Dr. Abernethy. In the last 2 years, she has had nominal consulting agreements with or received honoraria from (<$5000 annually) Novartis and Pfizer. Consulting with Bristol Meyers Squibb is pending in 2012, for role as Co-Chair of a Scientific Advisory Committee. Dr. Abernethy has paid leadership roles with American Academy of Hospice & Palliative Medicine (President-Elect) and Pillars4Life (Medical Director) in 2013. She has corporate leadership responsibility in Advoset (an education company that has a contract with Novartis) and Orange Leaf Associates LLC (an IT development company).
Footnotes
Conflict of Interest: No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the author(s) or on any organization with which the author(s) is/are associated.
References
- 1.American Cancer Society. Cancer Facts & Figures 2012. Atlanta: American Cancer Society; 2012. [Google Scholar]
- 2.Motzer RJ, Bander NH, Nanus DM. Renal-cell carcinoma. N Engl J Med. 1996;335:865–75. doi: 10.1056/NEJM199609193351207. [DOI] [PubMed] [Google Scholar]
- 3.Antonelli A, Cozzoli A, Zani D, et al. The follow-up management of non-metastatic renal cell carcinoma: definition of a surveillance protocol. BJU Int. 2007;99:296–300. doi: 10.1111/j.1464-410x.2006.06616.x. [DOI] [PubMed] [Google Scholar]
- 4.Hutson TE. Targeted therapies for the treatment of metastatic renal cell carcinoma: clinical evidence. Oncologist. 2011;16:14–22. doi: 10.1634/theoncologist.2011-S2-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gore ME, Larkin JMG. Challenges and opportunities for converting renal cell carcinoma into a chronic disease with targeted therapies. Br J Cancer. 2011;104:399–406. doi: 10.1038/sj.bjc.6606084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sosman JA, Puzanov I, Atkins MB. Opportunities and obstacles to combination targeted therapy in renal cell cancer. Clin Cancer Res. 2007;13:764s–9. doi: 10.1158/1078-0432.CCR-06-1975. [DOI] [PubMed] [Google Scholar]
- 7.Cella D. Beyond traditional outcomes: improving quality of life in patients with renal cell carcinoma. Oncologist. 2011;16:23–31. doi: 10.1634/theoncologist.2011-S2-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.EuroQol Group. Euro-Qol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16:199–208. doi: 10.1016/0168-8510(90)90421-9. [DOI] [PubMed] [Google Scholar]
- 9.Ware JE, Jr, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36), I: conceptual framework and item selection. Med Care. 1992;30:473–83. [PubMed] [Google Scholar]
- 10.Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85:365–76. doi: 10.1093/jnci/85.5.365. [DOI] [PubMed] [Google Scholar]
- 11.Harding G, Cella D, Robinson D, Jr, et al. Symptom burden among patients with renal cell carcinoma (RCC): content for a symptom index. Health Qual Life Outcomes. 2007;5:34. doi: 10.1186/1477-7525-5-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cella D, Yount S, Du H, et al. Development and validation of the Functional Assessment of Cancer Therapy - Kidney Symptom Index (FKSI) J Support Oncol. 2006;4:191–9. [PubMed] [Google Scholar]
- 13.Cella D, Li JZ, Cappelleri JC, et al. Quality of life in patients with metastatic renal cell carcinoma treated with sunitinib or interferon alfa: results from a phase III randomized trial. J Clin Oncol. 2008;26:3763–9. doi: 10.1200/JCO.2007.13.5145. [DOI] [PubMed] [Google Scholar]
- 14.Bukowski R, Cella D, Gondek K, et al. Effects of sorafenib on symptoms and quality of life: results from a large randomized placebo-controlled study in renal cancer. Am J Clin Oncol. 2007;30:220–7. doi: 10.1097/01.coc.0000258732.80710.05. [DOI] [PubMed] [Google Scholar]
- 15.Cella D, Yount S, Brucker PS, et al. Development and validation of a scale to measure disease-related symptoms of kidney cancer. Value Health. 2007;10:285–93. doi: 10.1111/j.1524-4733.2007.00183.x. [DOI] [PubMed] [Google Scholar]
- 16.Motzer RJ, Escudier B, Oudard S, et al. Efficacy of everolimus in advanced renal cell carcinoma: a double-blind, randomised, placebo-controlled phase III trial. Lancet. 2008;372:449–56. doi: 10.1016/S0140-6736(08)61039-9. [DOI] [PubMed] [Google Scholar]
- 17.U. S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, et al. Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockville, MD: U.S. Department of Health and Human Services, Food and Drug Administration; 2009. [Google Scholar]
- 18.Cella D, Paul D, Yount S, et al. What are the most important symptom targets when treating advanced cancer? A survey of providers in the National Comprehensive Cancer Network (NCCN) Cancer Invest. 2003;21:526–35. doi: 10.1081/cnv-120022366. [DOI] [PubMed] [Google Scholar]
- 19.Cella DF, Tulsky DS, Gray G, et al. The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol. 1993;11:570–9. doi: 10.1200/JCO.1993.11.3.570. [DOI] [PubMed] [Google Scholar]
- 20.Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of The Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5:649–55. [PubMed] [Google Scholar]
- 21.Cella D, Rosenbloom SK, Beaumont JL, et al. Development and validation of 11 symptom indexes to evaluate response to chemotherapy for advanced cancer. J Natl Compr Canc Netw. 2011;9:268–78. doi: 10.6004/jnccn.2011.0026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Escudier B, Szczylik C, Hutson TE, et al. Randomized phase II trial of first-line treatment with sorafenib versus interferon alfa-2a in patients with metastatic renal cell carcinoma. J Clin Oncol. 2009;27:1280–9. doi: 10.1200/JCO.2008.19.3342. [DOI] [PubMed] [Google Scholar]
- 23.Motzer RJ, Hutson TE, Tomczak P, et al. Overall survival and updated results for sunitinib compared with interferon alfa in patients with metastatic renal cell carcinoma. J Clin Oncol. 2009;27:3584–90. doi: 10.1200/JCO.2008.20.1293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Sternberg CN, Davis ID, Mardiak J, et al. Pazopanib in locally advanced or metastatic renal cell carcinoma: results of a randomized phase III trial. J Clin Oncol. 2010;28:1061–8. doi: 10.1200/JCO.2009.23.9764. [DOI] [PubMed] [Google Scholar]
- 25.Fairclough DL, Cella DF. Functional Assessment of Cancer Therapy (FACT-G): non-response to individual questions. Qual Life Res. 1996;5:321–9. doi: 10.1007/BF00433916. [DOI] [PubMed] [Google Scholar]
- 26.Brunelli C, Costantini M, Di Giulio P, et al. Quality-of-life evaluation: when do terminal cancer patients and health-care providers agree? J Pain Symptom Manage. 1998;15:151–8. doi: 10.1016/s0885-3924(97)00351-5. [DOI] [PubMed] [Google Scholar]
- 27.Justice AC, Rabeneck L, Hays RD, et al. Sensitivity, specificity, reliability, and clinical validity of provider-reported symptoms: a comparison with self-reported symptoms. J Acquir Immune Defic Syndr. 1999;21:126–33. [PubMed] [Google Scholar]
- 28.Nekolaichuk CL, Bruera E, Spachynski K, et al. A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med. 1999;13:311–23. doi: 10.1191/026921699675854885. [DOI] [PubMed] [Google Scholar]
- 29.Stephens RJ, Hopwood P, Girling DJ, et al. Randomized trials with quality of life endpoints: are doctors’ ratings of patients’ physical symptoms interchangeable with patients’ self-ratings? Qual Life Res. 1997;6:225–36. doi: 10.1023/a:1026458604826. [DOI] [PubMed] [Google Scholar]
- 30.Strömgren AS, Groenvold M, Sorensen A, et al. Symptom recognition in advanced cancer: a comparison of nursing records against patient self-rating. Acta Anaesthesiol Scand. 2001;45:1080–5. doi: 10.1034/j.1399-6576.2001.450905.x. [DOI] [PubMed] [Google Scholar]
- 31.Butt Z, Peipert J, Webster K, et al. General population norms for the Functional Assessment of Cancer Therapy–Kidney Symptom Index (FKSI) Cancer. 2013;119:429–37. doi: 10.1002/cncr.27688. [DOI] [PMC free article] [PubMed] [Google Scholar]