An exploratory analysis of the “Was it worth it?” questionnaire as a novel metric to capture patient perceptions of cancer treatment

Abstract

Objectives

Asking “Was it worth it?” (WIWI) potentially captures the patient perception of a treatment’s benefit weighed against its harms. This exploratory analysis evaluates the WIWI questionnaire as a metric of patients’ perspectives on the worthwhileness of cancer treatment.

Methods

A three-item WIWI questionnaire was assessed at end-of-treatment in cancer patients on the COMET-2 trial (NCT01522443). WIWI items were evaluated to determine their association with quality of life (QOL), treatment duration, end-of-treatment reason, patient-reported adverse events (PRO-AEs) and disease response.

Results

65 patients completed the questionnaire. 40(62%)/16(25%)/9(14%) patients replied yes, uncertain, and no to “Was it worthwhile for you to receive the cancer treatment given in this study?” [Item 1], respectively; 39(60%)/12(18%)/14(22%) to “If you had to do it over again, would you choose to have this cancer treatment?” and 40(62%)/14(22%)/11(17%) to “Would you recommend this cancer treatment to others?” Patients responding yes to Item 1 remained on treatment longer than those responding uncertain/no (mean 23.0 versus 11.3 weeks, p<0.001). Patients responding uncertain/no to Item 1 discontinued treatment due to AEs more frequently than those responding yes (36% versus 7.5%, p=0.004) and demonstrated meaningful decline in QOL from baseline (−2.5 versus −0.2 mean change, p<0.001). Associations between WIWI responses and most PRO-AEs or treatment efficacy did not reach statistical significance.

Conclusions

Patients who responded affirmatively on WIWI items remained on therapy longer, were less likely to stop treatment because of AEs and demonstrated superior QOL. The WIWI may inform clinical practice, oncology research, and value frameworks.

Summary (Precis):

The “Was it worth it?” is a promising tool for identifying treatments that cancer patients consider worthwhile. Patients who responded affirmatively remained on therapy longer and demonstrated superior quality of life.

Introduction

Understanding the totality of a patient’s experience with a given cancer therapy is complex, requiring capture of the patient’s perspective on the desired effects of cancer therapy (e.g., tumor shrinkage, greater longevity, improved symptoms and quality of life [QOL]) against its undesired effects (e.g., adverse events [AEs], treatment intensity, schedule and cost). Metrics that capture patients’ own assessment of risks and benefits of treatment are lacking and would be highly informative to other patients and clinicians. After reviewing a treatment plan, patients frequently ask their oncologists if what they will endure will be worthwhile. This is especially relevant to patients with advanced stage disease, where the goal of improved QOL and symptoms is often paramount and is weighed precariously against the risk of AEs and treatment intensity. Individual patients may weigh different factors in what defines and determines worthwhileness but ascertaining their perspective on this concept has value beyond a traditional patient satisfaction measure. It captures the reckoning of the totality of a patient’s experience, integrating multiple factors including toxicity, efficacy, tolerability, and financial burden, among others, in a complex decision.

A tool has been developed by the United States National Cancer Institute (NCI) to enable patient self-reporting of AEs in cancer clinical trials, the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).¹ Other potential metrics, such as the FACT-G GP5 item (“I am bothered by side effects of treatment”), similarly focus on AEs and fail to capture the patient’s view of the treatment overall.² AEs are just one of many factors contributing to patients’ perceptions of whether a treatment is worthwhile. A metric of the overall worthwhileness would encompass the numerous, difficult to capture, subjective considerations patients weigh when assessing their treatment experience. While what may be worthwhile undoubtedly varies among patients and clinical contexts, assessing this construct would illuminate the treatment experience in a novel and clinically meaningful way and be distinct from traditional patient satisfaction surveys. A concise metric that encapsulates whether a treatment is worthwhile from the patient’s perspective has the potential to enhance value frameworks^{3,4 5,6} by incorporating a summary measure of patients’ individual preferences and how they felt and functioned while on treatment.⁷

The Was It Worth It (WIWI) questionnaire was developed in the Mayo Clinic phase 1 research program to evaluate patient perceptions of the value of clinical trial participation.⁸ These questions were then adapted for use in the Alliance for Clinical Trials in Oncology.⁹ Although the WIWI has been widely integrated into Alliance trials to evaluate patients’ satisfaction with trial participation, it has not been tested for its ability to capture patient views on whether a cancer treatment was worthwhile. Therefore, in this exploratory study, we incorporated a modified WIWI assessment in a prospective, phase 3 advanced prostate cancer clinical trial to investigate patients’ view on the worthwhileness of a cancer treatment and to measure its relationship with key components of the patient-centered value assessment.

Methods

Patients and Methods

COMET-2 (NCT01522443) was a double-blind, placebo-controlled phase 3 registration-track superiority trial assessing the safety and efficacy of cabozantinib vs. mitoxantrone with prednisone in men with metastatic castration-resistant prostate cancer who had undergone 2+ prior lines of systemic treatment (clinical results reported elsewhere).¹⁰ The primary endpoint was pain response at 6 weeks and at 12 weeks. Secondary endpoints were bone scan response at week 12, and overall survival. Type 1 error was controlled across primary and secondary endpoints using sequential testing. The trial was discontinued early due to negative overall survival results of a companion phase III trial (COMET-1) which compared cabozantinib vs. placebo in this population.¹¹ There were exploratory endpoints which included WIWI survey responses. However, there was no formal control of type I error (i.e., each between arm comparison was to be carried out with a two-sided alpha of 0.05) across exploratory endpoints and results were considered descriptive. The exploration of the association between WIWI and other data in this trial was post hoc.

The WIWI questionnaire includes three items: (Item 1) Was it worthwhile for you to receive the cancer treatment given in this study? (Item 2) If you had to do it over again, would you choose to have this cancer treatment? and (Item 3) Would you recommend this cancer treatment to others? All COMET-2 participants were asked to complete the WIWI questionnaire that included three WIWI items at the time of going off treatment for any reason. These items contained the identical stems as previously developed except that they inquired about the worthwhileness of the cancer treatment rather than trial participation; the response options are “yes”, “uncertain” and “no”. Participants also completed 21 PRO-CTCAE items measuring 12 symptomatic AEs (insomnia, constipation, pain, fatigue, nausea, vomiting, diarrhea, rash, decreased appetite, numbness/tingling, mouth/throat sores, and shortness of breath)^1,12 and a single item global QOL assessment from the linear analogue self-assessment (LASA)¹³ at baseline and during treatment (week 3, week 6 and every 6 weeks thereafter; seven day recall). Radiographic tumor response was evaluated based on Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST).¹⁴

Statistical analysis

Each WIWI item was analyzed as a single item. Comparison of patient characteristics between those who completed the end-of-treatment (EOT) WIWI survey and those who did not used Wilcoxon rank sum and Fisher’s exact tests for continuous and categorical variables, respectively. Associations between WIWI items and continuous outcomes were assessed using analysis of variance F-tests. Associations between WIWI items and categorical outcomes were assessed using chi-squared tests. Analysis was first conducted considering each WIWI item as a categorical variable with three response options: yes, uncertain, and no. Analysis was subsequently conducted combining uncertain/no as a single category. Mean PRO-CTCAE scores were compared between groups using Wilcoxon rank sum tests. P-values <0.05 were considered statistically significant.

Results

Of 119 patients enrolled in COMET-2, 65 completed the EOT questionnaire which included the WIWI and PRO-CTCAE questions. All 65 patients completed all three items. Median age for the patients who completed the EOT questionnaire was 65 years (range 53–79), and performance status was 0–1 in 60 (92%) of these 65 patients. Relative to the 54 patients who did not complete the EOT survey, those who completed the EOT survey were descriptively more likely to be of better baseline performance status (proportion with ECOG PS 0–1 60/65 [92%] vs 45/54 [83%], p=0.16); significantly less likely to have received more than one line of prior cytotoxic therapy (as indicated by prior receipt of cabazitaxel, 21/65 [32%] vs 28/54 [52%], p=0.04) (Table 1); and were more likely to end treatment due to disease progression (42/65 [65%] vs 22/54 [41%], p=0.02). The EOT survey completion rate did not significantly differ between treatment arms (cabozantinib 31/61 [50.8%] vs mitoxantrone 34/58 [58.6%], p=0.46).

Table 1.

Baseline and other characteristics by completion status of the EOT survey

	Completed EOT survey (N=65)	Did not complete EOT survey (N=54)	P-value1
Treatment arm as randomized – n (%)			0.46
Cabozantinib	31 (47.7%)	30 (55.6%)
Mitoxantrone	34 (52.3%)	24 (44.4%)
Age in years – median (range)	65 (53–79)	66.5 (44–80)	0.17
Race – n (%)			0.75
White	55 (84.6%)	45 (83.3%)
Black	5 (7.7%)	6 (11.1%)
Other	5 (7.7%)	3 (5.6%)
Country – n (%)			0.01
Australia	11 (16.9%)	7 (13.0%)
Canada	8 (12.3%)	2 (3.7%)
United Kingdom	5 (7.7%)	16 (29.6%)
United States	41 (63.1%)	29 (53.7%)
ECOG Performance Status – n (%)			0.16
0–1	60 (92.3%)	45 (83.3%)
≥2	5 (7.7%)	9 (16.7%)
Prior Cabazitaxel – n (%)	21 (32.3%)	28 (51.9%)	0.04
End of treatment reason – n (%)			0.02
Disease progression	42 (64.6%)	22 (40.7%)
Adverse event	12 (18.5%)	13 (24.1%)
Other/unknown	11 (16.9%)	19 (35.2%)

EOT, end of treatment; ECOG, Eastern Cooperative Oncology Group

¹P-values are based on Wilcoxon rank-sum test for age and Fisher’s exact test for all other characteristics.

40 (62%), 16 (25%), and 9 (14%) patients replied “yes”, “uncertain”, and “no”, respectively, to “Was it worthwhile for you to receive the cancer treatment given in this study?” [Item 1]; 39 (60%), 12 (18%), and 14 (22%) to “If you had to do it over again, would you choose to have this cancer treatment?”; and 40 (62%), 14 (22%), and 11(17%) to “Would you recommend this cancer treatment to others?” There were no statistically significant differences between treatment arms (Item 1 rates of “yes”, “uncertain”, “no”: cabozantinib 19 [61.3%], 7 [22.6%], 5 [16.1%] vs mitoxantrone 21 [61.8%], 9 [26.5%], 4 [11.8%], p=0.88; Item 2: cabozantinib 19 [61.3%], 5 [16.1%], 7 [22.6%] vs mitoxantrone 20 [58.8%], 7 [20.6%], 7 [20.6%], p=0.89; Item 3: cabozantinib 20 [64.5%], 4 [12.9%], 7 [22.6%] vs mitoxantrone 20 [58.8%], 10 [29.4%], 4 [11.8%], p=0.21).

Patients who responded “yes” to Item 1 remained on treatment significantly longer than patients who responded “uncertain” or “no” (mean 23.0 vs 11.3 weeks, p<0.001). EOT reason was significantly associated with worthwhileness of cancer treatment from Item 1 (p=0.008). Specifically, of the 12 patients who discontinued treatment due to AEs, only 3 (25%) indicated the treatment was worthwhile. In contrast, of the 42 patients who discontinued treatment due to disease progression, 31 (74%) felt the treatment was worthwhile. Similar trends were observed for whether the patient would take the treatment again (p=0.01). Mean QOL at EOT significantly differed between patients responding “yes” and those responding “uncertain” or “no” for all three items (Figure 1, all p<0.05). Patients responding “uncertain” or “no” demonstrated meaningful decline in QOL from baseline, while QOL of patients responding “yes” to Item 1 remained stable (−2.5 versus −0.2 mean change, p<0.001) (Figure 1).

Figure 1.

“Was it Worth It?” responses and associations. (A) End-of-treatment “Was It Worth It?” item responses by mean overall quality of life at baseline at end of treatment, (B) mean change in overall quality of life from baseline to end of treatment, (C) mean treatment duration, and (D) end-of-treatment reason.

When examining associations between the PRO-CTCAE and WIWI at EOT, mean baseline-adjusted scores were numerically higher for 18 (85.7%) PRO-CTCAE items when comparing patients responding “uncertain” or “no” to those responding “yes” to Item 1; however, only four reached statistical significance (Supplementary Table 1). Similarly, in exploring the relationship of WIWI to treatment outcome, the proportion of patients considered responders by quantitative bone scan at 12 weeks was descriptively higher among those responding “yes” to Item 1 than those responding “uncertain” or “no”, but this did not achieve statistical significance (9/40 [22.5%] vs 3/25 [12.0%], p=0.34)

Relationships among the three items are displayed in Supplementary Figure 1. The large bands connecting the same responses across items demonstrate a high level of agreement in patient responses across items. Specifically, 31 (48%) patients indicated “yes” to all 3 questions and 17 (26%) patients indicated “uncertain” or “no” to all 3 questions. Among the remaining 17 patients with at least 1 “yes” response and at least 1 “uncertain” or “no” response, the question that was most often in disagreement was “would you choose this cancer treatment again?” (7 [41%] patients responded differently to this question as compared to the other two questions). The question that was least often in disagreement was Item 1 (“Was it worthwhile for you to receive the cancer treatment given in this study?”): 4 [24%] patients responded differently to this question as compared to the other two questions. Patients who felt the treatment was worthwhile were significantly younger than those who did not (mean age 63.4 [SD 5.0] vs 68.0 [SD 6.9], p=0.01). Similarly, patients who would recommend the treatment were significantly younger than those who would not (mean age 63.9 [SD 5.6] versus 67.3 [SD 6.6], p=0.05). Other baseline patient characteristics (race, country, performance status or prior cabazitaxel) were not related to WIWI item responses.

Discussion

There have been substantial recent efforts to quantify the value of cancer treatments, including the development of frameworks that balance benefits, risks and costs.¹⁵ Particular attention has been paid to bringing the patient experience into these tabulations. Although QOL, symptoms, or other patient-reported information can be considered, there have not been direct assessments of patients’ summation of risks and benefits of a given treatment in many of these frameworks. The findings of this exploratory study provide evidence of the potential meaningfulness of directly asking patients whether they feel a treatment was worthwhile. The WIWI may inform value frameworks to enable a more patient-centered approach. Additionally, in informing future patients about the perceptions of patients who previously received a given treatment, the WIWI may potentially aid treatment selection in clinical practice and evaluation of risk:benefit assessments of new treatments in cancer trials.

The high level of agreement among the three WIWI items in this initial evaluation suggests that although a 3-item scale might offer some additional precision, there is limited additional clinically relevant information beyond administering the first item “Was it worthwhile for you to receive the cancer treatment given in this study?” Therefore, in future analyses, just this single item could be administered to capture the patient perspective on the value of a treatment. This study demonstrates that patient-reports of “worthwhileness” are most strongly associated with treatment duration, the reason for ending treatment, and overall QOL, and less strongly with treatment efficacy (e.g., disease response) or toxicity (e.g., patient-reported AEs). Intriguingly, a substantial portion of patients who experienced AEs or disease progression indicated treatment was worthwhile. This reflects the inherently subjective nature by which patients weigh the risks and benefits of therapy and may indicate that some patients with advanced disease feel an attempt at treatment is worthwhile regardless. Future work should identify what factors patients consider and how they weigh them when assessing the value of treatment.

This analysis has several limitations. A small number of participants completed the EOT questionnaire. There was high drop out in this trial population composed of heavily pretreated patients with metastatic disease in both arms who were quite ill at baseline. Logistics at EOT are often challenging administratively and patients may face disappointment about needing to discontinue treatment, which may have contributed to the rate of missing data. The EOT survey in this study was also administered in a different modality than the on-study questionnaires (paper survey versus interactive voice response), and if the study coordinator became aware of this retroactively, they would not have been an opportunity to administer this survey in clinic. The reasons for ending treatment may impact patients’ ability or willingness to complete questionnaires, thus leading to informative missingness. In this study, we assessed for bias resulting from informative missingness by assessing differences in baseline patient characteristics and EOT reasons between those completing the EOT questionnaire and those who did not. As some differences were identified, bias cannot be excluded. The aim of this exploratory analysis was to evaluate associations involving these items rather than to show a difference between study arms, and there were sufficient data for the intended purpose. However, the small sample size led to reduced statistical power to detect associations with other metrics such as AEs and disease response. In future implementations of the WIWI, the questionnaire should be administered during treatment at a time point or multiple time points prior to anticipated dropout. Administering the WIWI earlier during treatment may provide a larger sample size and better representation of the trial population, as well as longitudinal assessment of worthwhileness. Continuing to administer it at EOT is still advisable to capture the patient’s complete evaluation of worthwhileness. Lastly, our study does not clarify what specific factors different patients may weigh in answering the WIWI questionnaire as this was not our goal. However, given the trends in response identified with treatment duration and discontinuation due to AEs in this initial evaluation, future work could include a qualitative study to understand the drivers of response among diverse patients in different clinical situations (early stage curable malignancy versus advanced setting, for example) performing a reckoning of their cancer treatment.

In conclusion, an exploratory study of the WIWI as a measure of the worthwhileness of cancer treatment will add to the existing body of evidence about the benefits, risks, and costs of cancer treatments from the patient perspective. Future work may focus on implementation in larger and more diverse samples of patients in different lines of cancer treatment, an alternative administration schedule to minimize missing data, and qualitative evaluation of the survey to better understand the concept of worthwhileness from the patient perspective. The findings of this study provide evidence of the potential meaningfulness of directly asking patients whether they feel a treatment was worthwhile.

Conclusions

In a brief and potentially single metric item, the WIWI questionnaire could potentially inform selection of cancer treatment and risk:benefit assessments of therapy. Further investigation of this tool in larger studies in different clinical situations in oncology trials and clinical practice is warranted.

Highlights.

• Metrics that capture worthwhileness of cancer treatment from the patient’s perspective are lacking. Asking patients “Was it worth it” (WIWI) may illuminate a patient’s perceptions on a cancer treatment’s benefits weighed against its harms.

• In this exploratory analysis, a three-item WIWI questionnaire was evaluated as a potential measure of patients’ perspectives on the worthwhileness of cancer therapy.

• Affirmative responses to WIWI questions were associated with remaining on cancer therapy longer, being less likely to stop due to adverse events (AEs), and higher quality of life (QOL) than those who responded negatively or with uncertainty. As such, the WIWI is a brief tool with the potential to provide information useful to clinical practice, oncology trials and value frameworks on a patient’s perspective on worthwhileness of cancer treatment.