Progression-free survival as a clinical trial endpoint in advanced renal cell carcinoma

Abstract

Traditionally, overall survival (os) has been considered the “gold standard” for evaluating new systemic oncologic therapies, because death is easy to define, is easily compared across disease sites, and is not subject to investigator bias. However, as the available options for continuing therapy increase, the use of os as a clinical trial endpoint has become problematic because of the increasing crossover and contamination of trials. As a result, the approval of promising new therapies may be delayed.

Many clinicians believe that progression-free survival (pfs) is a more viable option for evaluating new therapies in metastatic and advanced renal cell carcinoma. As with all endpoints, pfs has inherent biases, and those biases must be addressed to ensure that trial results are not compromised and that they will be accepted by regulatory authorities. In this paper, we examine the issues surrounding the use of pfs as a clinical trial endpoint, and we suggest solutions to ensure that data integrity is maintained.

1. INTRODUCTION

Traditionally, overall survival (os) has been the “gold standard” for the evaluation of new systemic therapies for cancer because death is easily defined, is clinically important, is not subject to investigator or assessment bias, and can easily be compared across diseases and disease sites. But the use of os as a primary endpoint has significant disadvantages (Table i). Planned and unplanned trial crossover to investigational and other therapies occurs in approximately 50%–60% of patients in trials studying metastatic renal cell carcinoma (mrcc)^1,2. Many patients are now being treated with sequential therapies, and os as a trial endpoint will not accurately reflect the benefits of the investigational drug with multiple lines of treatment³. The use of os requires long-term follow-up, and because of a lower event rate, it also requires a larger number of patients than progression-free survival (pfs) does. The requirement for a larger number of patients may increase trial costs, particularly when the disease has a long natural history, and it may also impede researchers from opening new trials.

table i.

Comparison of progression-free survival and overall survival as an endpoint for clinical trials in metastatic renal cell cancer

Progression-free survival		Overall survival
Pros	Cons	Pros	Cons
May increase access to effective therapies earlier, because results are available sooner or endpoint is reached sooner	Affected by the timing of assessments, a factor that must be addressed by using symmetrical evaluation schedules and methods in both arms of the clinical trial	Not affected by the timing of assessments	Potentially delays access by patients to new active treatments because of waits for results of studies
Assuming similar power and desired minimal difference detected, fewer patients are needed on study because progression events occur earlier and more frequently than do death events for overall survival	Investigator assessment can be prone to bias (investigator bias), a factor that must be addressed by using strict criteria for response evaluation and by performing that evaluation in a central, independent, blinded fashion	Death endpoint is easy to determine because the definition is not arbitrary and is not prone to investigator and assessment bias	Assuming similar power and desired minimal difference detected, more patients are needed on study because death events occur later and less frequently than do progression events for progression-free survival
Because of the need for fewer patients and shorter follow-up, progression-free survival is associated with lower study costs and shorter durations	Considered an imperfect surrogate by Health Canada advisors	The “gold standard”	Because of the need for more patients and longer follow-up, overall survival is associated with higher study costs and longer durations
Not affected by issues of crossover or contamination by subsequent active therapies	May not be a direct measure of clinical benefit—for example, progression is often asymptomatic and may not always be clinically relevant	Death is always clinically significant	Results may be contaminated by subsequent agents and diluted by crossover, which may lead to no demonstrable difference in overall survival
May be a benefit in and of itself	Definitions of progression vary and may not reflect quality of life, pain, or performance status		Does not address patient quality of life and ability to work and function
	Cannot compare well with other disease groups who do not use progression-free survival	Comparison across disease sites is straightforward
	Response Evaluation Criteria in Solid Tumors may not be applicable to cytostatic agents	Is relevant for all treatment modalities
Increased progression-free survival may lead to increased quality of life	More difficult to do quality-adjusted life years analysis	Easier to do quality-adjusted life years analysis
	Greater likelihood of missing progression data than death data	Death data may be captured at a later date from other records
	Progression-free survival benefit may not translate into a benefit in overall survival and thus must be validated in each disease setting; also, large differences in progression-free survival may be required to be clinically significant	Ensures that treatments do not shorten patient lifespan from other causes

These clinical endpoint issues have been highlighted in trials examining vascular endothelial growth factor (vegf) receptor inhibitors in mrcc. In many mrcc tumour cells, vegf is overexpressed, probably because of deregulation of the VHL (Von Hippel–Lindau) tumour suppressor gene, leading to increased angiogenesis^4,5. New therapies in mrcc inhibit angiogenesis by targeting vegf, its receptor, or related receptors (Table ii)^1,2,6–14. Although most of these agents have shown clear signs of clinical activity in earlier studies and are being widely used, the optimal endpoints to use in the evaluation of these treatments in phase iii trials remain controversial.

table ii.

Summary of angiogenesis inhibitor trials

Reference	Pts (n)	Treatment arms	Phase and line of treatment	Progression-free survival		Overall survival		Comments
				Median (months)	hr (95% ci)	Median (months)	hr (95% ci)
Bevacizumab (Bev)
Escudier et al., 20076, 20097	327 322	Bev+ifnα vs. placebo+ifnα	Phase iii First line	10.4 5.5	0.57 (0.45 to 0.72) p<0.0001	Bev+ifn arm: 23.3 ifn+placebo arm: 21.3	0.86 (0.72 to 1.04) p=0.1291	Frequency of subsequent therapy not reported, but did happen
Rini et al., 20081, 20098	369 363	Bev+ifnα vs. ifnα	Phase iii First line	8.5 5.2	0.71 (0.61 to 0.83) p<0.0001	18.4 17.4	0.86 (0.73 to 1.01) Stratified log-rank p=0.069	No crossover allowed, but subsequent systemic therapy in about half of patients
Yang et al., 20039	39 37 40	Bev 10 mg vs. bev 3 mg vs. placebo	Phase ii Second line	4.8 3.0 2.5	10 mg: 2.55 p<0.001 3 mg: 1.26 p=0.053	nr p=ns
Everolimus
Motzer et al., 200810	277 139	Everolimus vs. placebo	Phase iii Second or third line	4.9 1.9	0.33 (0.25 to 0.43) p<0.001	14.8 14.4	0.87 (0.65 to 1.17) p=0.177	Of the patients in the placebo arm, 81% crossed over to treatment
Pazopanib
Sternberg et al., 201011	290 145	Pazopanib vs. placebo	Phase iii First or second line	Overall:		nr		The objective response rate was 30% with pazopanib compared with 3% with placebo (p<0.001) Patients on the control arm were allowed to cross over to the treatment arm on progression Too early to evaluate overall survival data
				9.2 4.2	0.46 (0.34 to 0.62) p<0.0001
				Treatment-naïve:
				11.1 2.8	0.40 (0.27 to 0.60) p<0.0001
				Cytokine-pretreated:
				7.4 4.2	0.54 (0.35 to 0.84) p<0.001
Sorafenib
Escudier et al., 200712	451 452	Sorafenib vs. placebo	Phase iii Second line	5.5 2.8	0.44 (0.35 to 0.55) p<0.001	17.8 15.2	0.88 (0.74 to 1.04) p=0.15	The unplanned crossover of patients from the placebo arm to active treatment compromised survival data
Szczylik et al., 200713	97 92	Sorafenib vs. ifnα	Phase ii First line	5.7 5.6	0.88 (0.61 to 1.27) p=0.50	nr
Sunitinib
Motzer et al., 20072	375 375	Sunitinib vs. ifnα	Phase iii First line	11 5	0.54 (0.45 to 0.64) p<0.001	26.4 21.8	0.82 (0.67 to 1.00) p=0.05	Patients on the control arm were allowed to cross over to the treatment arm on progression
Temsirolimus
Hudes et al., 200714	209 210 207	Temsirolimus vs. temsirolimus+ifnα vs. ifnα	Phase iii First line	3.8 3.7 1.9	nr	10.9 8.9 7.3	Temsirolimus: 0.73 (0.58 to 0.92) p=0.008
							Temsirolimus+ifnα: 0.96 (0.76 to 1.20) p=0.70

Pts = patients; hr = hazard ratio; ci = confidence interval; ifnα = interferon-alfa; nr = not reported; ns = nonsignificant.

Table ii illustrates the consistent demonstration, in recent mrcc trials, of a statistically significant and clinically relevant improvement in pfs without significant differences in os. This observation is most likely an artifact of crossover to more active therapy in the investigational arm or subsequent treatment with active agents at the time of progression. Even though these studies have failed to meet the important endpoint of os, vegf inhibitors are now—based on obvious clinical benefits—the mainstay of therapy in patients with mrcc. For those reasons, a reassessment of the clinical trial endpoints that will be used to assess future treatments for mrcc, as well as other cancers, is necessary.

2. PFS AS A SURROGATE ENDPOINT

A surrogate endpoint for os is any intermediary endpoint that strongly correlates with or predicts os. Ideally, the surrogate should capture clinically relevant events and have a clearly defined, easily measurable start and finish. It should be observable at a time early enough to establish, with sufficient power, an earlier indication of efficacy. Prentice criteria require that a surrogate endpoint be a prognostic factor and that, after a patient achieves the surrogate endpoint, the time to disease-specific mortality be independent of the treatment received¹⁵. Oncologists and patients both believe that pfs is often a reliable surrogate endpoint for os that meets those requirements. They also believe that the duration of stable disease may be an important additional measure.

Progression-free survival is the length of time during and after a treatment that a patient living with a disease does not get worse. In most clinical trials, it is defined as the time from randomization to progression (defined by either radiologic or clinical measures). In many adjuvant oncology trials, pfs is already an accepted endpoint. Examples include studies of panitumumab in patients with colorectal cancer and of aromatase inhibitors in patients with breast cancer.

As a surrogate, pfs has a number of potential strengths and weaknesses (Table i).

Detection of progression can be affected by the timing and frequency of assessments; by measurement bias (particularly in tumours that are smaller or in which margins are not clear, or when one metastatic lesion is growing while another is shrinking); or by patient attrition, including missed or late evaluations. Such issues do not affect os. Investigator bias can also influence the time at which progression is recorded. For example, if earlier identification of progression will enable crossover to another active treatment, or if later identification will ensure continued treatment with an agent believed to be beneficial, investigator bias may unintentionally influence reporting of progression.

Finally, pfs may not be a direct measure of clinical benefit, and because progression is often asymptomatic, it may not always be clinically relevant. Large differences in pfs may therefore be required to show clinical relevance. It is possible, especially when the improvement is relatively minor, that a pfs benefit may not translate into an os benefit. Scientific advisors have therefore suggested to Health Canada that pfs is an imperfect surrogate for os¹⁶.

2.1. PFS in mRCC

Most clinicians believe that, in mrcc patients treated since the introduction of targeted therapies, a real change and improvement in os has occurred. An era-by-era comparison of population-based data from British Columbia¹⁷ and Alberta¹⁸ showed a large gain in os for patients treated in the sunitinib era compared with those treated in the cytokine era. That outcome means that the pfs benefit demonstrated in the phase iii sunitinib clinical trial translated into an os benefit in a population-based setting².

Further validation was provided by an abstract presented by Delea et al.¹⁹ at the 2009 annual meeting of the American Society for Clinical Oncology in which 29 systemic therapy trials for mrcc were combined in a meta-analysis in which time to progression and pfs were compared with os. In general, a positive difference in pfs was correlated with a positive difference in os, and no improvement in pfs was correlated with no improvement in os. This association was strongest for immunotherapy trials; it was not as clear in trials of targeted therapy, possibly because of study crossover and contamination. In immunotherapy trials, pfs could be used to predict os, clearly establishing the link between the two. Biologically, the relationship between pfs and os is more likely linked to the natural history of the disease than to the treatment or treatments received, unless the treatment received might be expected to somehow reduce the amount of time a patient spends between progression and eventual death. For that reason, it is appropriate to accept that the findings of Delea et al. support the validity of pfs as an appropriate outcome measure and a possible surrogate for os in mrcc. One criticism of this group’s work is that most of the authors worked in the pharmaceutical industry, suggesting a need for independent validation.

More support for the validation of pfs as a surrogate endpoint in mrcc was provided by an analysis done by Heng et al.²⁰. Data for 1158 patients from multiple participating countries were analyzed to determine whether pfs could be used to predict os. Landmark analyses at various time points compared the survival of patients who progressed with that of patients who did not progress. With a median follow-up of 30.6 months, patients who progressed at 3 months had a median os of 7.8 months compared with 23.6 months for patients who did not progress at that time (log rank test: p < 0.0001), and patients who progressed at 6 months had a median os of 8.6 months compared with 26 months for patients who did not progress at that time (p < 0.0001). For the patients who progressed at 3 months and at 6 months, the hazard ratios for death after adjustment for adverse prognostic factors were 3.05 (95% confidence interval: 2.42 to 3.84) and 2.96 (95% confidence interval: 2.39 to 3.67) respectively. Similar results were demonstrated with landmark analyses at 9 months and at 12 months, and in the bootstrap validation.

2.2. PFS in Other Types of Cancer

A recent Canadian Oncology Societies workshop addressed the issue of pfs as a surrogate endpoint for metastatic colorectal cancer (mcrc)²¹. Attendees noted that patient outcomes have improved with the evolution of mcrc treatment options, and that it has become impossible to use os as an endpoint for individual lines of therapy on account of the number of subsequent treatments that patients usually receive. A recent study showed a correlation between treatment with 3 or more sequential agents and longer os, which has led researchers to believe that os is no longer an appropriate endpoint for mcrc²¹. In a separate meta-analysis of 39 trials, pfs was studied to determine if it could be used as a surrogate endpoint for os in mcrc²². In first-line chemotherapy trials, improvements in pfs were strongly associated with improvements in os²¹. Because of those meta-analyses, pfs is now the recognized standard in colorectal cancer trials.

In a study of castrate-resistant prostate cancer, data from 1296 men participating in nine different trials were pooled in a meta-analysis to determine if pfs could be used to predict os²³. Men who did not progress at 3 and 6 months lived twice as long (adjusted hazard ratio: 2.0) than men who did progress. Although the authors concluded that there was a correlation between pfs and os in prostate cancer, the use of prostate-specific antigen as a marker of progression is more problematic and more unreliable than is the use of radiography in other solid tumours.

Data from eleven randomized breast cancer trials involving 3953 patients were pooled to determine if a surrogate for os could be found in metastatic breast cancer²⁴. In that trial, the rank correlation coefficient between pfs and survival showed moderate correlation, but the authors concluded that pfs was not an adequate surrogate for os.

In some situations, failure to validate pfs as a surrogate for os may come as a result of the surrogate not being valid or as a reflection of the statistical modelling used. At a conference on surrogate endpoints held in Montreal in 2006, 5 different researchers were given the same dataset from ten mcrc trials and were then asked if pfs was a valid surrogate for os²⁵. Each of the 5 researchers used a different meta-analytic approach to combine the data, and each came back with a different response: 2 reported that, as a surrogate, pfs was an adequate measure; 2 concluded that pfs was not an adequate surrogate for os; and 1 stated that reliable interpretation of the data was not possible. Those findings underscore the need for a rigorous methodology to validate pfs as a surrogate for os.

3. ACCEPTANCE OF PFS BY REGULATORY AND FUNDING AUTHORITIES

In the absence of os data, pfs data present notable challenges to those who make reimbursement decisions. However, the U.S. Food and Drug Administration (fda) and the European Medicines Agency (ema) have accepted pfs as an endpoint to support drug approval in several types of cancer^26,27. The fda guidelines²⁶ state that, for regular approval, it is critical that the applicant show direct evidence of clinical benefit or improvement in an established surrogate endpoint for clinical benefit. Whether an improvement in pfs represents a direct clinical benefit or a surrogate for clinical benefit depends on the magnitude of effect and the risk–benefit profile of the new treatment compared with available therapies. The fda guidelines note that formal validation of pfs as a surrogate for survival for many malignancies can be difficult and that the data are usually insufficient to allow for a robust evaluation of the correlation between effects on pfs and on os. The ema guideline²⁷ states that precise estimates of os may not be needed for approval in situations in which a large effect on pfs, an extended expected survival after progression, or a clearly favorable safety profile is observed. If os is not considered to be an appropriate endpoint, then the ema expects the study protocol to set out the reasons that endpoints such as survival benefit or symptom control cannot be used as a primary measure of patient benefit.

In Canada, Health Canada gives market authorization for new treatments when, based on their evaluation of submitted safety, efficacy, and quality data concerning the potential benefits and risks of the treatment, it can be concluded that the benefits of the treatment outweigh the risks and that the risks can be mitigated. However, the provinces and territories manage their own health care budgets and hence make their own decisions on reimbursement. Traditionally, Health Canada has required os data for the approval of new agents after regulatory review. However, they acknowledge that os data may not always be obtainable, in which case, market authorization may be granted under the policy of conditional authorization. Conditional authorization provides for earlier access to new drugs based on promising evidence of clinical effectiveness and an acceptable safety profile.

Health Canada is concerned that pfs does not necessarily measure how a patient feels, functions, or survives, and that it is therefore not a direct measure of clinical benefit. Trial design issues (such as asymmetry in the frequency of assessments between study arms) have also been problematic. For example, a lack of documentation of patient compliance with the visit schedule does not allow for proper assessment of potential bias in the treatment effect estimate. Scientific advisors have suggested to Health Canada that, although pfs is considered, at best, to be an imperfect surrogate for os, if the magnitude of the benefit is large, pfs may be an indication of clinical benefit¹⁶. Health Canada has also noted that many patients and clinicians feel that pfs is associated with improved quality of life (qol); however, they are rarely provided with study data that confirm this link. To summarize, although Health Canada remains convinced that os is the best endpoint to evaluate the efficacy of oncology drugs, they consider pfs to be an endpoint that may demonstrate clinical benefit in certain situations. The use of pfs as a surrogate for os must be separately validated for each disease site and for each drug mechanism of action.

To be funded, new treatments usually must be considered cost-effective. Conventional cost-effectiveness analyses compare the relative clinical benefits of a new agent with those of the funded alternative or alternatives, and weighs the relative cost of the new agent against its relative clinical benefits. A common type of analysis yields a ratio called cost per quality-adjusted life-year (qaly) gained. The qaly calculation considers both the survival and the qol associated with treatment, and it is applicable across disease areas. In the absence of os data, a “cost per progression-free year” could be estimated that would make comparisons with other treatments for other conditions based on cost per qaly difficult and potentially invalid. In addition, a unit of progression-free year gained in mrcc may not be comparable to a unit of progression-free year gained in another type of cancer.

It might be possible to calculate a cost per qaly if pfs data are combined with data providing estimates of survival after progression. Such data are often found in natural history studies of mrcc. This method of calculating a survival benefit requires a study of mrcc that is contemporary, relevant, and of high quality. The relevance is evaluated based on the similarity of the patients who received the new treatment and progressed in the trial compared with the “progressive disease” patients in the natural history study. In a contemporary natural history study, the treatments subsequently used for progressing patients are similar to those being used in the current context, and the similarity is evaluated based on whether any breakthrough has been achieved in new salvage treatment that has significantly changed survival after progression. Extrapolating clinical benefit beyond the trial period is subject to potential bias, and validation often will be required to demonstrate face validity of the analysis.

4. DISCUSSION AND FUTURE STEPS

Life expectancy for mrcc patients has increased greatly since the introduction of angiogenesis inhibitors in the first- and second-line settings, and because clinical trials are now taking place in third-line settings, investigators believe that pfs is an appropriate study endpoint—a clinically relevant and valid surrogate for os in evaluating new treatments in these patients. New active agents are being introduced with increasing frequency. Although cure is rarely possible, patients appear to be benefiting from administration of sequential therapies. Because it could be considered unethical to deny subsequent treatment to patients who could benefit from such treatment, most of these patients will receive multiple lines of therapy, preventing the collection of os data that is relevant to the investigational agent.

Many of the current clinical trials in mrcc now allow for crossover of patients from the control arm to the experimental arm—a circumstance that dilutes differences in os. Even without crossover, the availability of other active targeted therapies applied after progression (or after study exit because of toxicity on the clinical trial) could further contaminate overall survival analyses, especially if the subsequent agents are very similar to, if not the same as, the agent under investigation (for example, vegf inhibitors). These trials are likely to find little difference in os between the trial arms because most of the patients on the trial will receive the alternative agent on study or an alternative agent off study. Indeed, in studies in which crossover is allowed, the results of the trial show more the effect of early compared with late exposure to the agent. For example, in the randomized phase iii study evaluating everolimus in comparison with placebo, 91% of patients crossed over to the active treatment, most after only 2 cycles of placebo therapy¹⁰. In general, it is believed that patients and investigators find study participation more acceptable if the possibility of crossover is present, even without unblinding, when progression or toxicity has occurred. Improvement in os from a single treatment is now almost impossible to demonstrate in mrcc.

A group of clinician leaders in the field of mrcc in Canada recently met to discuss and debate the data set out in the present paper. The overall consensus was that rigorously defined pfs is a valid surrogate endpoint for mrcc. However, new clinical trials must be designed to have sufficient power to detect a clinically relevant improvement in pfs as the primary endpoint (Table iii). The os data still need to be collected to ensure that overall lifespan is not being detrimentally affected and to provide historical data about the usefulness of sequential treatments as research proceeds. Data on qol should be collected where clinically relevant. To address regulatory concerns about multiple bias issues, trials must be designed with appropriate controls through blinding and randomization and with standardized, centralized, unbiased blinded radiologic assessments. A standardized definition of pfs is required, and that definition should be applied to all future trials, as should a standardized method of assessing the trials. To influence reimbursement decisions, it will be necessary to be able to estimate qalys, the standard outcome in cost–effectiveness analysis, from pfs data, which will require relevant data on survival after progression and qol data for progression-free patients and for the same patients after they progress. For this calculation, qol data has to be collected using validated instruments for measuring health utility, such as the EQ-5D (EuroQol Group, Rotterdam, Netherlands)²⁸.

table iii.

Quality requirements for use of progression-free survival as a primary endpoint

Randomized, blinded study

Defined and consistent assessments of response in each treatment arm

Central radiology review

Clinically relevant absolute gain in progression-free survival

Improvement in progression-free survival supported by other endpoints

Interval between progression and death expected to be 6 months or more

Sufficient data collected to evaluate impact on overall survival at a later date

In summary, accumulating data support the validity and acceptability of pfs as an outcome measure of therapeutic effectiveness in mrcc. Future studies may use pfs as a primary endpoint, but to be accepted by regulatory agencies, the results will have to come from clinical trials that have accepted a common definition of pfs and that, having been designed with a sound and rigorous methodology, demonstrate a treatment effect on pfs of sufficient magnitude to be considered clinically relevant. As a clinical trial endpoint, pfs remains most useful and credible if it is concordant with, and linked to, os and qol data.