Using quality-adjusted progression-free survival as an outcome measure to assess the benefits of cancer drugs in randomized-controlled trials: case of the BOLERO-2 trial

Abstract

The aim of this study is to estimate the quality-adjusted progression-free survival (QAPFS) as an effectiveness measure for the treatment arms of the BOLERO-2 trial. For each treatment arm of the trial, QAPFS was estimated by multiplying the overall health utility weights associated with progression-free survival (PFS) (accounting for utility decrements associated with the adverse events of treatments) by the corresponding mean PFS time. Health utility data were obtained from the literature, while mean PFS times were estimated through a survival analysis of the reconstructed individual patient data of the BOLERO-2 trial. PFS (robust mean, (95 % robust confidence interval)) was 44.73 weeks (41.03; 48.43) for Everolimus + Exemestane and 22.98 weeks (19.88; 26.08) for Placebo + Exemestane. The QAPFS (robust mean, (95 % robust confidence interval)) for the treatment arms of the trial was 30.09 (27.60; 32.58) for Everolimus + Exemestane and 16.27 (14.07; 18.46) for Placebo + Exemestane, respectively. Using QAPFS as an outcome measure provides a complete picture of the benefit induced by the treatment arms of the BOLERO-2 trial. The benefit of Everolimus + Exemestane over Placebo + Exemestane observed in the trial is maintained in this analysis. The approach and estimates obtained as part of our analysis can serve as a basis for cost effectiveness analyses of the treatment arms of the BOLERO-2 trial.

Introduction

Decisions to grant market authorization for new drugs and decisions about what treatments to recommend to patients are driven by safety and efficacy data obtained from randomized-controlled clinical trials (RCTs) [1]. Two issues arise when attempting to make informed decisions about the value of drugs based only on safety and efficacy data. The first issue relates to whether or not to use interim outcome over final outcome measures. The second issue has to do with the failure of both interim and final outcome measures to fully capture patients’ quality of life (QoL), which is of paramount importance for optimal patient care. These two issues are. respectively, discussed in the following paragraphs.

The benefits of drugs in RCTs are assessed against interim (e.g., biological markers) and/or final outcome (e.g., mortality) measures. Final outcome measures are the usual and recommended endpoints in RCTs as they directly impact patients’ life expectancy. However, in some cases, cancer treatments for instance, assessing the benefits of treatments using an interim outcome measure, such as progression-free survival (PFS), may be preferred over a final outcome measure, overall survival (OS) for example. This situation occurs when progression of the disease requires that treatments be modified (e.g., titration, addition of second-line treatments, etc.) to optimize clinical patient outcomes [2]. Under this scenario, it would be a challenge to measure the real effect of first-line treatments on OS due to synergistic, antagonistic, or additive effects that would bias the causal relationship between first-line treatments and OS.

Safety and efficacy data alone are insufficient to allow clinicians to make informed decisions as to whether they should recommend newly marketed treatments to their patients. The reason is that these data do not relate directly to patient’s QoL. An example would be a treatment that prolongs the life expectancy of a cancer patient by 5 years. This information is not sufficient since the new treatment could cause the patient to live five additional years in extreme pain or prevent him/her from being able to carry out daily activities. A potential solution to this issue is to combine safety and efficacy data with quality of life data as done in cost-utility analyses. In this paper, an illustration of combining safety and efficacy data with quality of life data is given with the breast cancer trials of oral everolimus-2 (BOLERO-2) [3].

The BOLERO-2 is a double-blind, phase 3 trial that compared Everolimus plus Exemestane (n = 485) versus placebo plus Exemestane (n = 239) [3]. Postmenopausal women with advanced hormone receptor positive breast cancer were included in the study. The trial demonstrated that Everolimus plus Exemestane significantly prolonged PFS. Even though PFS seems to be an appropriate measure to estimate the efficacy of Everolimus plus Exemestane (treatment arm 1) compared to Exemestane alone (treatment arm 0), it fails to capture the QoL of patients living in that state. To adequately capture the effects of treatment arms 0 and 1 on patients’ quality of life, a complete outcome measure such as quality-adjusted progression-free survival (QAPFS) can be used. The current paper is aimed at illustrating the estimation of the QAPFS induced by the treatment arms of the BOLERO-2 trial.

Methods

For each treatment arm of the trial, QAPFS was estimated by multiplying the overall health utility weights associated with the PFS state (taking into consideration disutilities associated with the adverse events of treatments) by the corresponding mean PFS time. It is worthwhile emphasizing that the estimation of utility values, per norm, is based on mean instead of median values. As such, the BOLERO-2 trial only reported median values for PFS time. These median values represent the time until which 50 % of the cohort of patients followed up have not progressed yet. Using median instead of mean values would underestimate the QAPFS for both treatment arms (see Table 2). To overcome this issue, mean PFS times were estimated through the survival analysis of the reconstructed individual patient data (IPD) of the BOLERO-2 trial. Health utility data were obtained from the literature.

Table 2.

Mean progression-free survival time associated with the treatment arms of the BOLERO-2 trial

Treatment arm	Records	N.max	N.start	Events	Rmean	Se(rmean)	0.95 LCI	0.95 UCI	Median	0.95 LCL	0.95 UCL
0	239.00	239.00	239.00	197.00	22.98	1.58	19.9	26.08	14.10	12.10	18.10
1	485	485	485	310	44.73	1.89	41.03	48.43	34.4	30.2	37.3

N.max maximum number of patients assigned to the treatment arm, N.start number of patients assigned to the treatment arm, rmean restricted mean with upper limit, se(rmean) standard error of the restricted mean, 0.95 LCI Lower limit of the 95 % confidence interval, 0.95 UCI Upper limit of the 95 % confidence interval

The mean of survival is restricted to the time before the last censoring

The restricted means and their standard errors are based on truncated estimators

Estimating utility and disutility weights for each treatment arm of the BOLERO-2 trial

Different approaches can be used to estimate utility and disutility weights for the population studied in the BOLERO-2 trial. One approach is to convert health-related quality of life (HRQOL) data collected as part of the trial [4] into health utility increments and decrements by mapping HRQOL data from the EORTC QLQ-C30 onto the EuroQoL-5 dimension (EQ-5D)-derived utilities. A limitation with this approach is that it requires access to the original HRQOL data collected, which is rarely granted by the promoter of the trial. Even in case of data availability, mapping data from quality of life questionnaires onto derived utilities involve the use of regression models such as Ordinary Least Squares (OLS). In several studies, using mapping algorithm, OLS have been reported to overpredict mean EQ-5D index in patients exhibiting fairly poor health [5–8]. A preferred alternative to this approach is to use utilities for metastatic breast cancer disease states directly obtained from health utility instrument such as the time trade-off or standard gamble.

In this study, utility increments and decrements associated with disease states described in the BOLERO-2 trial were derived from a study by Lloyd and colleagues [9]. The authors of this study elicited societal preferences for health states describing different metastatic breast cancer disease states (stable on treatment, responding disease, and progressive disease) and six common toxicities in the United Kingdom (Table 1).

Table 1.

Health utility/disutility weights

Utilities	Weighted average values	Original values (distributions)	Source
Base state–Stable disease	0.715	0.715	Lloyd et al. [9]
Disutility
Grade 3 and 4 adverse events
Arm 1
Stomatitis	0.01208	0.151 (8 %)	Lloyd et al. [9]; Baselga et al. [3]
Anemia	0.00906	0.151 (6 %)a	Lloyd et al. [9]; Baselga et al. [3]
Dyspnea	0.00604	0.151 (4 %)a	Lloyd et al. [9]; Baselga et al. [3]
Hyperglycemia	0.00604	0.151 (4 %)a	Lloyd et al. [9]; Baselga et al. [3]
Fatigue	0.0046	0.115 (4 %)	Lloyd et al. [9]; Baselga et al. [3]
Pneumonitis	0.00453	0.151 (3 %)a	Lloyd et al. [9]; Baselga et al. [3]
Arm 0
Stomatitis	0.00151	0.151 (1 %)	Lloyd et al. [9]; Baselga et al. [3]
Anemia	0.00151	0.151 (1 %)a	Lloyd et al. [9]; Baselga et al. [3]
Dyspnea	0.00151	0.151 (1 %)a	Lloyd et al. [9]; Baselga et al. [3]
Hyperglycemia	0.00151	0.151 (1 %)a	Lloyd et al. [9]; Baselga et al. [3]
Fatigue	0.00115	0.115 (1 %)	Lloyd et al. [9]; Baselga et al. [3]
Pneumonitis	0	0.151 (0 %)a	Lloyd et al. [9]; Baselga et al. [3]

Treatment arm 0: Exemestane only; Treatment arm 1: Everolimus plus Exemestane

^aTo obtain the utility decrements for adverse events, for which values were not available in the literature, we multiplied the highest utility decrement value available (0.151) by the proportion of patients experiencing these adverse events

Estimating the mean progression-free survival time for each treatment arm of the BOLERO-2 trial

The mean PFS times were estimated following the survival analysis of the reconstructed individual patient data (IPD) of the final Kaplan–Meier (KM) PFS curves of the BOLERO-2 trial [10, 11]. Our analysis was based on local PFS assessment, which happened to be the primary endpoint of the BOLERO-2 trial. Efficacy of anti-cancer drugs is assessed locally in routine clinical practice. Therefore, the reconstruction of IPD was based on local PFS data to reflect real life clinical practice. The reconstruction of IPD, providing a good approximation of the original IPD data, was performed using an algorithm developed by Guyot and colleagues (algorithm implemented in R statistical package) [12]. This entailed, first, the extraction of the coordinates of the points composing final KM PFS curves of the BOLERO-2 trial [10, 11], using a computer digitization program. Second, a new dataset comprising a sequence of survival time intervals matching the trial follow-up time, the upper and lower bounds of the extracted coordinates matching the survival time intervals, and the number of individuals at risk for each interval was created. Third, the algorithm was applied to the new dataset to find numerical solutions to the inverted final KM survival equations, based on available information on the number of events and number of patients at risk [12]. Once the IPD were reconstructed, the restricted mean PFS times were derived by adding a statement to the aforementioned algorithm [see page 3 of the algorithm (accessible at http://www.biomedcentral.com/content/supplementary/1471-2288-12-9-s1.pdf), under the section “Find Kaplan–Meier estimates”] as follows: print(survfit(Surv(IPD[,1],IPD[,2]) * 1), print. rmean = TRUE).

Estimating quality-adjusted progression-free survival for each treatment arm of the BOLERO-2 trial

The QAPFS is a product of the net utility (the difference between utility associated with the PFS state and disutility from adverse events) and the estimated mean PFS time of each treatment arm using the following equation:

$${\mathit{QAPFS}}_i=(U_i-{dU}_i)\times {\mathit{rmean}}_i,$$

where i is the treatment arm, QAPFS represents the quality-adjusted progression-free survival, U_i is the utility for the stable (PFS) state and dU_i is the disutility due to the adverse events associated with the treatment arm, and rmean_i is the mean PFS time. Note that the dU_i was a weighted average of the common grade 3/4 adverse events reported in the BOLERO-2 trial.

Results

The reconstructed data including the mean PFS time for Everolimus plus Exemestane and Exemestane alone are shown, respectively, in Table 2. PFS [robust mean; (95 % robust confidence interval)] was 44.73 weeks (41.03; 48.43) for Everolimus plus Exemestane and 22.98 weeks (19.88; 26.08) for Exemestane. Patients who received Everolimus plus Exemestane lived 21.75 weeks longer and free of progression than those who received Exemestane alone. The net utility value for treatment arm 0 and 1 was estimated at 0.71 and 0.67, respectively.

The QAPFS (robust mean, (95 % robust confidence interval)) for the treatment arms of the trial was 30.09 (27.60; 32.58) for Everolimus plus Exemestane and 16.27 (14.07; 18.46) for Exemestane alone, respectively.

Discussion

We initially argued that using PFS as an outcome measure to compare treatment strategies for advanced hormone receptor positive breast cancer is incomplete as it fails to account for the quality of life of patients living in that disease state. To address this issue, we suggested that researchers can estimate the QAPFS of treatments as an effectiveness measure. The BOLERO-2 trial served as an illustrative case. The QAPFS for patients in the BOLERO-2 trial was estimated. The results showed that patients treated with Everolimus plus Exemestane had longer PFS time than those who were treated with Exemestane alone. Patients treated with Everolimus plus Exemestane had relatively better QAPFS time compared to patients treated with Exemestane alone. The direction and magnitude of the benefits of Everolimus plus Exemestane over Exemestane observed in the trial were maintained in this analysis.

Other studies have also relied on QAPFS to compare cancer treatments. A recent Dutch study evaluated the QAPFS of gefitinib versus relevant doublet chemotherapies in advanced non-small cell lung cancer (NSCLC) patients with epidermal growth factor receptor mutation-positive stage IIIb/IV NSCLC during the progression-free state [2]. The rationale for using the QAPFS in this study was based on the fact that QAPS takes account of the additional health-related quality of life (HRQOL) benefits associated with the treatments [2]. The authors of this study found that the QAPFS was longer for gefitinib than for the doublet chemotherapies, confirming the progression-free survival benefit of first-line gefitinib in advanced NSCLC EGFR M + patients over standard care. Another study evaluated trade-offs between QoL and survival improvement, in terms of quality-adjusted survival outcomes, by comparing patients with advanced breast cancer treated with single-agent mitoxantrone and combination of cyclophosphamide, methotrexate, 5-fluorouracil, and prednisone (CMFP) [13]. In that study, QAPFS for each group was the product of its mean utility score and the area under its time-to-progression curve truncated at 24 months. The study found that QAPFS was significantly longer with CMFP, but quality-adjusted OS was not significantly different.

The current study has some strengths and limitations that deserve highlighting. In terms of strengths, our study uses a protocol that allows for the reconstruction of IPD data based on published Kaplan–Meier curves. The reconstruction of IPD is particularly useful when IPD from RCTs are not readily available. In this study, the estimation of QAPFS needed median PFS values obtained from the BOLERO-2 to be converted into mean values. This was accomplished using a validated algorithm from Guyot et al. [12], which was updated by adding a statement to the original algorithm. In addition, the estimation of QAPFS took into consideration the utility decrements associated with the toxicities induced by the competing treatments. However, the main limitation of this study is that our analysis is only based on one RCT. The availability of additional data obtained from other head-to-head comparison would have helped in further confirming the estimates obtained in this study. That said, the approval of Everolimus in combination with Exemestane was done in light of the same single data source, BOLERO-2 trial.

Overall, the present findings suggest that using QAPFS as the outcome measure provides a better assessment of the benefits induced by the treatment arms of the BOLERO-2 trial. The QAPFS estimates obtained as part of our analysis can be used as measure of effectiveness in future cost effectiveness studies.