Challenges, complexities, and considerations in the design and interpretation of late-phase oncology trials

Abstract

Optimal management of cancer patients relies heavily on late-phase oncology randomized controlled trials. A comprehensive understanding of the key considerations in designing and interpreting late-phase trials is crucial for improving subsequent trial design, execution, and clinical decision-making. In this review, we explore important aspects of late-phase oncology trial design. We begin by examining the selection of primary endpoints, including the advantages and disadvantages of using surrogate endpoints. We address the challenges involved in assessing tumor progression and discuss strategies to mitigate bias. We define informative censoring bias and its impact on trial results, including illustrative examples of scenarios that may lead to informative censoring. We highlight the traditional roles of the log-rank test and hazard ratio in survival analyses, along with their limitations in the presence of non-proportional hazards and an introduction to alternative survival estimands, such as restricted mean survival time or MaxCombo. We emphasize the distinctions between the design and interpretation of superiority and non-inferiority trials, and compare Bayesian and frequentist statistical approaches. Finally, we discuss appropriate utilization of phase II and phase III trial results in shaping clinical management recommendations, evaluating the inherent risks and benefits associated with relying on phase II data for treatment decisions.

Introduction

Late-phase oncology randomized controlled trials (RCT) form the primary basis for approval of new cancer therapies by regulatory agencies. Therefore, it is essential to comprehensively understand the design of these trials, recognize potential biases that may arise during the study, and address subsequent challenges in interpreting the findings. This knowledge is invaluable for both trialists and clinicians in order to enable informed decision-making and empower patients to actively participate in treatment recommendations. Furthermore, given the substantial costs and time investment involved in the development of novel therapies,¹ it is imperative to optimize all available resources for the benefit of patient care. In this article, we present an overview of key topics in late-phase clinical trial design and execution (Table 1). Topics covered include the selection of primary endpoints, identification of potential sources of bias, considerations for superiority and non-inferiority trials, exploration of alternative survival endpoints, and the role of Bayesian statistical inference in cancer trials. Lastly, we discuss controversies on the appropriate utilization of late-phase trial data to inform regulatory decisions.

Table 1:

Key concepts affecting late-phase trial design, execution, and interpretation

Topic	Background	Issues and Impact on Late-Phase Trials	Takeaway
Primary endpoint selection and Surrogate endpoints	• Surrogate endpoints (e.g. progression-free survival) are increasingly used as primary endpoints in late-phase trials and as the basis for regulatory approval of new therapies	• Surrogate endpoints often do not correlate with overall survival or quality-of-life endpoints • Heightened risk of approving therapies on the basis of surrogate endpoints that may have uncertain benefit	• Exercise caution when interpreting surrogate endpoint trial data particularly in the absence of a validated correlation with overall survival or quality of life
Assessment of tumor progression	• Tumor progression is usually assessed with an existing standard such as the Response Evaluation Criteria in Solid Tumors (RECIST)	• Inter-rater variability exists despite standardized definitions of progression • Open-label trials in which tumor progression is assessed by the local investigator may be at risk of bias	• Use of blinded independent central review (BICR) of imaging may reduce inter-rater variability but introduces financial and logistical costs and the risk of different biases
Informative censoring	• Informative censoring bias occurs when patients who are censored are more or less likely to experience an event than patients who remain on study • Informative censoring may be more likely in studies requiring active follow-up or imaging scans to assess the study endpoint	• Informative censoring may distort trial results, particularly those with surrogate endpoints • For example, informative censoring may artificially improve the progression-free survival results of an experimental therapy that is more toxic than standard of care if experimental arm patients who discontinue treatment were likely to progress with minimal added follow-up	• Trialists can reduce the risk of informative censoring by assessing overall survival, ensuring the control arm receives standard-of-care therapy, and requiring additional imaging assessments after treatment discontinuation
Nonproportional hazards	• In survival analyses, the validity of the hazard ratio depends upon the proportional hazards assumption, i.e. a constant hazard ratio over the follow-up period	• Non-proportional hazards (NPH) causes uncertainty in interpretating the hazard ratio and reduces the power of the log-rank test • NPH, present 1 in 4 late-phase oncology trials, may result in unclear or inappropriate interpretation of trial results	• Trialists should routinely assess for proportional hazards • Trialists should consider alternative survival estimands that do not depend upon proportional hazards (e.g. restricted mean survival time)
Superiority and Non-inferiority trials	• Superiority trials determine if one therapy is more effective than another • Non-inferiority trials determine if one therapy is not meaningfully worse than another • Superiority and non-inferiority trials differ in key respects (e.g. sample size requirements, analysis cohort), and lack of superiority does not imply noninferiority	• Superiority trials with negative results have in practice been used to justify non-inferiority (e.g. CONVERT, RTOG 0538), leading to challenges in the clinical interpretation of such results	• Clinicians should resist the temptation to extrapolate non-inferiority from a negative superiority trial • Trialists must continue to carefully weigh the implications of selecting between superiority and non-inferiority trial designs
Frequentist vs. Bayesian statistical inference	• Oncology trials traditionally rely on frequentist statistical inference (i.e. p-values, 95% confidence intervals), as opposed to Bayesian statistical inference, which incorporates data obtained prior to or during the study period	• P-values do not describe the probability of the study hypothesis being true or false and do not provide information about effect size or clinical significance • Rare diseases are difficult to study due to sample size requirements that outpace enrollment • Frequentist trial results, in practice, are interpreted in the context of prior data, bioplausibility, etc.	• Bayesian trials allow for transparency in the assumptions used to interpret trial results • Bayesian adaptive trials incorporate data obtained during the study and may produce more efficient trials that are useful when enrollment is difficult
Interpretation of Phase II trial results	• Phase III trials are the gold standard for practice-changing evidence but are costly and time-consuming, and the quality of evidence from phase II and III studies varies by trial design, endpoint selection, and the strength of trial results • Phase II data has been used to change the standard of care (e.g. RTOG 0529)	• Phase II trials meeting their primary endpoint often do not replicate those results in Phase III • Phase II studies often lack randomization • Phase II studies may underestimate or overestimate effect sizes based upon how the studies are powered	• Phase III studies should continue to be the mainstay evidence used to change the standard of care • Caution should be exercised when using Phase II data to inform practice changes

Primary endpoint selection

Selecting an appropriate primary endpoint is a critical consideration in late phase RCTs with time-to-event endpoints. Traditionally, overall survival, defined as the time from diagnosis or intervention to death from any cause, has been the gold standard for evaluating the efficacy of novel therapies. However, modern cancer trials increasingly utilize intermediate endpoints, also known as surrogate endpoints, such as progression-free survival, relapse-free survival, or disease-free survival.²

The adoption of surrogate endpoints provides an alternative approach to assess the effectiveness of experimental therapies. Surrogate endpoints offer the potential to expedite trial completion³ and reporting,⁴ accelerating the introduction of new treatments for patients. Unlike overall survival, surrogate endpoints are less affected by treatment crossover at the time of disease progression, which may complicate the detection of differences in overall survival between therapies.⁵

Despite potential advantages of surrogate endpoints, it is essential to exercise caution when interpreting trial results relying on surrogate endpoints. A positive outcome measured with a surrogate endpoint does not guarantee an overall survival benefit⁶ or improved quality of life outcomes.⁷ In fact, novel treatments associated with improved progression-free survival may be associated with greater cost and toxicity compared to the standard of care.⁸ In many cases, the correlation between surrogate endpoints and overall survival may not have been adequately validated before regulatory approval.^9,10 Several factors contribute to the lack of association between surrogate and overall survival endpoints, including the diluting effect of crossover at the time of progression,^5,11 heterogeneity in progression determination and evaluation, and informative censoring that may bias the results of surrogate endpoint comparisons.¹² In the subsequent sections, we will delve into a more detailed discussion of some of these possibilities and their implications in the context of late-phase design and interpretation.

Assessing tumor progression: the role of blinded independent central review

Trials that rely on radiographic progression of an existing lesion often use a standardized definition of disease progression, such as the Response Evaluation Criteria in Solid Tumors.¹³ However, it is important to recognize that delaying tumor progression does not always correlate with prolonged survival or improved patient quality of life.¹⁴ Furthermore, despite standardized criteria to define progression, even for specific agents such as immunotherapy, inter-rater variability exists,^15,16 and the potential for investigator bias in open-label trials, however wellintentioned, may also confound results.¹⁷

To address these concerns, trials are usually recommended to incorporate blinded independent central review of imaging scans (BICR) to mitigate inter-rater variability, especially when blinding investigators or patients to the randomization is not feasible. In trials utilizing BICR, an independent evaluator, unaware of the randomization or other patient information, determines whether progression has occurred. However, the implementation of BICR presents financial and logistical challenges for trials, and it is uncertain whether BICR fully eliminates systematic bias.¹⁶ In fact, BICR may introduce different forms of bias in survival analysis. For instance, if a local investigator identifies progression before the central reviewer, the patient may discontinue the study. However, this patient would then be censored before the determination of progression using BICR, despite being more likely to experience progression at the next BICR assessment compared to other patients.¹⁸ This phenomenon is an example of informative censoring, a type of bias which we will explore in greater detail in the subsequent section.

Informative censoring biases

Informative censoring is another factor that can weaken the association between surrogate endpoints and overall survival. Censoring is an important aspect of Kaplan-Meier survival analyses, which are frequently used in trials with time-to-event endpoints. Censoring occurs when a patient does not experience the event of interest during the follow-up period and is assumed to be non-informative, meaning their risk of experiencing the event should be similar to others in the study with continued follow-up. However, the presence of informative censoring can distort survival analyses by artificially influencing outcomes in a particular study arm.¹⁹

Surrogate endpoints are more susceptible to informative censoring biases. For example, consider an experimental treatment that has a higher toxicity profile compared to the control treatment. In this scenario, patients receiving the experimental therapy may be more likely to discontinue or withdraw from the study before experiencing progression, leading to informative censoring related to the surrogate endpoint. However, these patients would not be censored in the assessment of overall survival, as determining the date of death is usually more straightforward than the date of progression, which often requires specific follow-up and imaging scans.

Similarly, in trials with suboptimal control arm therapies,²⁰ some argue that patients may be more prone to early study discontinuation due to disappointment with their randomization. Healthier or better-resourced patients may be more willing and able to drop out, leading to the premature removal of patients with more favorable prognostic outcomes from the control arm. Consequently, this may negatively impact the observed outcomes in the control arm, resulting in an exaggeration of the benefit associated with the experimental therapy.²¹

Further research is needed to better understand the impact of informative censoring on surrogate endpoint outcomes. In the interim, routinely assessing for the presence of informative censoring and ensuring that patients in the control arm have access to standard-of-care therapies may help mitigate the potential influence of informative censoring on surrogate endpoint outcomes. Additionally, exploring alternative surrogate endpoints that incorporate treatment harm from toxicity or early study discontinuation, such as treatment failure analysis,²² may prove valuable.

Proportional hazards violations and alternative survival measures

The log-rank test is the most commonly utilized statistical test for evaluating treatment efficacy in RCTs. Log-rank evaluates whether there is a difference in the probability of survival at any time point²³ between multiple groups. However, the log-rank test does not provide information about effect size or which treatment is better. To address this, the hazard ratio is typically calculated, which compares the hazard functions of the experimental and control study arms. The hazard ratio of Cox regression relies on the assumption of proportional hazards (PH), which holds that the relative hazards for comparison groups remain constant over the follow-up period.

In oncology trials, the PH assumption is routinely violated,^24,25 and under conditions of non-proportional hazards (NPH), the hazard ratio has an unclear interpretation.²⁶ Trials assessing immunotherapy appear to have particularly higher rates of non-proportional hazards,^27,28 as immunotherapy often exhibits a delayed treatment effect, presenting challenges to the interpretation of newer trials that increasingly incorporate these agents.^29,30 Perhaps the most recognizable scenario of NPH occurs with the crossing of treatment effects, also known as crossing hazards, in which the direction of effect changes over the course of the follow-up period.³¹ A decaying treatment effect, in which the treatment effect appears to decline over time, may also lead to non-proportional hazards.

While the log-rank test does not require the PH assumption, it may have reduced power to detect treatment differences under NPH conditions.^32–34 Alternatives survival estimands have been proposed to overcome these limitations. One such alternative is the the difference in restricted mean survival time (RMST),^35,36 which represents the integral under the survival function up to a specific time point. Reconstructed patient-level data from the CELESTIAL trial³⁷ (NCT01908426) is provided in Figure 1 to illustrate the RMST. RMST provides a patient-centric description of treatment efficacy and the magnitude of treatment benefit in a single test.³⁸ Mean survival time may offer a more precise measure of survival than median survival,³⁹ as it incorporates information across the follow-up period rather than a single point in time. RMST is particularly useful when events are scarce, as the hazard ratio loses precision in such situations.⁴⁰ Another test, MaxCombo, incorporates the standard log-rank test and a series of additional weighted log-rank tests designed for different scenarios of NPH, such as with delayed treatment effect, before selecting the best-performing test.²⁷. However, the use of multiple weighted tests may increase the risk of type 1 error.⁴¹ Other proposed alternatives include the integrated log-rank test,⁴² flexible parametric cure models,⁴³ or Bayesian inference.⁴⁴ These alternative approaches offer valuable options to address NPH and may provide a more comprehensive and valid interpretation of treatment efficacy in oncology trials.

Figure 1:

Reconstructed Kaplan-Meier curves of overall survival from the CELESTIAL trial (NCT01908426) comparing cabozantinib to placebo in patients with hepatocellular carcinoma (A), with the restricted mean survival time of the cabozantinib arm (B) depicted by the area under the curve up to 36 months.

Superiority and Non-inferiority Trials: Differences in design and interpretation

Phase 3 RCTs are most often designed to assess the superiority of experimental therapy compared to the standard of care. However, in certain scenarios where the new therapy offers advantages such as convenience, cost-effectiveness, or reduced toxicity, non-inferiority trials are employed to determine if the new therapy is not meaningfully worse than the standard of care. Non-inferiority radiotherapy trials comparing hypofractionated regimens to conventional fractionation feature prominently in the literature for breast cancer,⁴⁵ prostate cancer,^46,47 and other indications.

There are several key differences in the design of trials assessing superiority versus non-inferiority. First, non-inferiority trials generally require larger sample sizes than superiority trials. Determining the sample size for a non-inferiority assessment involves establishing the non-inferiority margin, which represents the acceptable level of detriment of the experimental therapy compared to the standard therapy. While the non-inferiority margin should always be set below the expected benefit from the standard therapy, defining this margin is often quite controversial and challenging given a degree of inherent subjectivity and interpretation.^48,49 Detecting smaller treatment differences necessitates larger sample sizes, contributing to increased cost and resources required for non-inferiority trials. Secondly, the statistical analyses used in superiority and non-inferiority assessments may differ. In superiority trials, intention-totreat analyses are typically preferred to provide a conservative estimate of treatment effect by accounting for factors that may obscure treatment differences, such as patient withdrawal, protocol deviations, or non-compliance.^50,51 In non-inferiority assessments, intention-to-treat analyses may increase the likelihood of observing a non-inferior result through the same washing out of treatment differences; thus, per-protocol analyses are more likely to also be utilized in the statistical analysis plan of non-inferiority trials.⁵²

The choice between superiority and non-inferiority designs can have profound consequences on the interpretation of trial results. For example, the superiority of twice-daily fractionation compared to daily fractionation for limited-stage small cell lung cancer was established by RTOG 8815⁵³. However, the study did not account for differences in the biologically effective dose of radiation between the two arms, resulting in sub-optimal biologically effective dose in the daily fractionation arm. The subsequent CONVERT study ⁵⁴ was a superiority RCT comparing the standard-of-care BID regimen to 60 Gy in 30 daily fractions, but did not demonstrate that 60 Gy in 30 daily fractions was superior to the BID regimen. Similarly, the CALGB 30610 / RTOG 0538 superiority RCT comparing 70 Gy given once daily versus standard of care BID did not demonstrate improved overall survival associated with the dose-escalated arm,⁵⁵ concluding that BID fractionation remains the standard of care. Despite these results, many clinicians continue to use the daily fractionation scheme,⁵⁶, possibly due to logistical challenges for patients and physicians associated with the BID regimen. It is important to note that there are clear methodological pitfalls in inferring that the two arms of either CONVERT or RTOG 0538 have similar efficacy, as neither trial was powered to assess non-inferiority. Nevertheless, it is interesting that the next generation of phase III clinical trials for limited stage small cell lung cancer, including NRG-LU005 (NCT03811002) and ADRIATIC, allows for either fractionation regimen, as does the NCCN guidelines.⁵⁷ This uncertainty highlights the challenges faced by trialists in the trial design process and that clinicians face in interpreting imperfect data from well-accrued negative superiority-design studies. Overall, careful consideration of the trial design and proper interpretation of results are crucial to informing treatment decisions and advancing patient care for both superiority and non-inferiority trials.

Bayesian statistical inference in cancer trials

Traditionally, oncology trials have employed a frequentist approach to statistical inference, utilizing p-values and 95% confidence intervals to compare treatment efficacy. A p-value indicates the probability of obtaining the observed result, plus more extreme results, if the null hypothesis (i.e., no effect of a treatment on survival) were true. The conventional threshold for rejecting the null hypothesis is a p-value less than 0.05. However, there are numerous issues associated with the use and interpretation of p-values.^58–60 Importantly, the p-value does not indicate the probability of the study hypothesis being true or false, nor does it describe the likelihood that the data were produced solely by random chance. Large p-values do not provide evidence of no effect, and p-values do not offer insights into effect size or clinical significance.

In contrast to the frequentist approach of assigning probabilities to data, Bayesian inference assigns probabilities to hypotheses by incorporating priors, composed of information known or believed about the efficacy of a given treatment that is independent of the study.^61,62 This information can come from published data, expert opinion, or biological plausibility. Bayesian analyses typically utilize multiple potential priors, including a non-informative prior, an “enthusiastic” prior that may assign a higher probability of seeing a clinically meaningful effect, and a “skeptical” prior that assigns a lower probability of seeing a clinically meaningful effect, to account for the subjectivity and potential differences in priors. ^63,64 Priors are then combined with the likelihood, representing data from the study participants, to obtain the posterior. All three (prior, likelihood, and posterior) are represented by probability distributions. Bayesian inference makes the assumptions underlying the interpretation of trial data generated with frequentist approaches more transparent. Bayesian analyses are particularly valuable when few patients can be feasibly enrolled and analyzed on the study, as is often the case in the study of rare diseases.⁶⁵ Additionally, Bayesian inference may be used in adaptive clinical trials, incorporating accumulating data during the trial to make more efficient inferences^66,67. Yet, despite the potential advantages to Bayesian inference, its utilization in phase 3 cancer trials has been limited.⁶⁸ Challenges such as unfamiliarity of the approach among trialists and statisticians, the need for coordination in updating and communicating trial results in adaptive designs,⁶⁹ and uncertainty among clinicians in interpreting and applying results from Bayesian-designed trials may contribute to the slow adoption of Bayesian approaches. Nonetheless, Bayesian methods have the potential to enhance trial efficiency, interpretation, and even validity, especially for adaptive trial designs. Addressing familiarity and logistical challenges could facilitate wider implementation of Bayesian approaches in late-phase cancer trials.

Interpretation of Phase II and Phase III Trials

Phase III clinical trials are widely regarded as the gold standard for establishing and changing the standard of care. However, conducting and completing phase III trials present numerous challenges, including significant costs, time investment, and resource requirements. Many phase III trials fail to accrue patients, and in fact one in 15 phase III trials do not reach publication.⁷⁰ Furthermore, the relevance of the clinical question at the trial’s initiation may be affected by advances in other fields by the time the trial concludes. An example is RTOG 1112, randomizing patients with unresectable hepatocellular carcinoma to sorafenib, the standard of care at the time of trial initiation in 2012, with or without stereotactic body radiotherapy.^71,72 While overall survival and progression-free survival benefits were demonstrated with the addition of stereotactic body radiotherapy to sorafenib, the optimal management of unresectable hepatocellular carcinoma remains uncertain, as standard-of-care systemic therapy changed from sorafenib to atezolizumab plus bevacizumab between RTOG 1112 initiation in 2012 and trial reporting in 2022.⁷³ These challenges raise important questions for oncologists, payors and national guideline committees: is a phase III trial necessary to change the standard of care? When is it justifiable to change the standard of care based on phase II study findings? Under what circumstances, if any, is it even desirable to do so? How do the endpoints, design and clinical context of the clinical question impact these considerations?

Advocates of changing standard of care based on phase II data emphasize the importance of timely delivery of new therapies to patients, particularly in settings with high mortality rates. An accelerated pathway for FDA drug approval has been in place for three decades, leading to approval of dozens of drugs based on phase II data without phase III confirmation.^74,75 Additionally, improvements in central quality assurance across surgical and radiotherapy trials have enhanced the credibility of modern phase II trials compared to historical phase III studies.⁷⁶ Moreover, the application and extrapolation of evidence to patient care should be necessarily influenced by study design. It is worth noting that 70% of oncology drugs approved by the FDA rely on phase III trials with surrogate primary endpoints, rather than what many argue are more patient-centered endpoints, such as overall survival or quality of life.^77,78 Should the normative of oncology be to change standard of care based on phase III studies only improving overall response rate, or phase II trials improving overall survival? In situations where a well-conducted phase II randomized study demonstrates a survival benefit (especially if survival benefit occurs in the setting of cross-over), there may be a perceived lack of equipoise and enthusiasm among oncologists and patients for a randomized phase III study.⁷⁹ Other design factors, such as patient selection criteria and the treatments used in the control arm, can significantly impact interpretability.^20,80–84 Therefore, it is essential to consider whether a poorly designed phase III trial should hold greater evidentiary weight than a well-conducted phase II study. Furthermore, phase II data often inform changes in the standard of care for rare conditions where conducting and completing a phase III trial may be unfeasible. An example is the phase II RTOG 0529, which definitively changed the standard of care approach to chemoradiation in anal squamous cell carcinoma from three-dimensional conformal radiotherapy to intensity-modulated radiotherapy.⁸⁵ The RTOG 0529 also illustrates two even more controversial considerations for changing standard of care based on phase II data: RTOG 0529 did not meet its primary endpoint (acute grade ≥ 2 gastrointestinal and genitourinary toxicity), and RTOG 0529 was a single-arm trial comparing outcomes to historical controls from another study (RTOG 9811).⁸⁶

While RTOG 0529 successfully changed standard of care, there are strong criticisms regarding the use of phase II data to drive such changes. One key concern is that single-arm phase II designs lack a controlled comparison between the experimental treatment and standard of care. These designs are typically intended to estimate effect sizes for powering subsequent phase III studies, but when they replace the phase III trial, the absence of randomization introduces several uncontrolled statistical biases emerging that can adversely impact interpretation. One well-known bias is the Will Rogers phenomenon, wherein improvements in diagnostic modalities lead to stage migration, resulting in apparent improvements in clinical outcomes even if the experimental treatment is not genuinely superior to the historical treatment.⁸⁷ Furthermore, even if a phase II study is randomized, there is risk of serious underestimation or overestimation of the effect size depending on how investigators power the study to control for type 1 or type II errors. Specifically, powering a non-inferiority design with a phase II sample size carries the greatest risk for type II errors. It is important to recognize that the majority of drugs meeting the primary endpoint in phase II trials ultimately fail to demonstrate primary endpoint efficacy in phase III, underscoring the criticality of careful consideration when using phase II data to inform changes in the standard of care.⁸⁸

Ultimately given the above, while there may be exceptional circumstances where changing the standard of care based on phase II data is reasonable, phase III trials should continue to serve as the gold standard for guiding standard of care decisions. Oncologists should prioritize the most evidence-based care for their patients. Innovative trial designs, such as adaptive phase II/III platforms or multi-arm umbrella studies, are gaining popularity due to time and cost advantages compared to the traditional sequence of phase II followed by phase III trials.^89,90 . Further exploration and study of these innovative design platforms are warranted.

Conclusions

Late-phase randomized controlled trials play a crucial role in introducing therapies that have the potential to enhance and lengthen the lives of patients with cancer. It is imperative to approach the design, execution, and interpretation of these studies with careful deliberation. This article has highlighted several considerations and potential challenges associated with late-phase cancer trials. To improve the interpretation of trial results and their application to patient care, ongoing education of oncologists is necessary. Furthermore, continued research is warranted to gain a deeper understanding of these issues and to propose innovative solutions that can enhance cancer clinical trial design. Addressing these concerns can continue to advance the field of oncology and optimize late-phase trials for the benefit of cancer patients.