Point/counterpoint: randomized versus single-arm phase II clinical trials for patients with newly diagnosed glioblastoma

In this article, we attempt to delineate the pros and cons of single-arm versus randomized phase II studies in patients with newly diagnosed glioblastoma (GBM), taking into account such factors as (i) the availability of appropriate controls, (ii) the interpretability of the resulting data, (iii) the goal of rapidly screening many novel agents using as few patients as necessary, (iv) utilization of limited financial and patient resources, and (v) maximization of patient participation in these studies.

Historical Perspective

Phase II trials are typically considered middle development studies and address questions related to clinical outcome and tolerability. The overall goal is to obtain preliminary estimates of the likelihood a patient will benefit from treatment as well as the likelihood a patient may suffer a serious adverse effect from treatment. This informs the potential risk-benefit evaluation of the drug upon which decisions are made to further evaluate the drug in a definitive phase III trial or to stop further study of the regimen. Traditionally, phase II drug studies in oncology assess adverse events using the Common Terminology Criteria for Adverse Events scale and usually do not have formal rules for the determination of whether the regimen is deemed too toxic; this is often left to clinical judgment of the acceptability of the adverse event profile and severity in the context of an estimate of the potential benefit; there is more tolerance of adverse events for regimens that potentially have greater clinical benefit.

Early in the era of oncology drug development, there were few effective agents across all cancer types. The primary role of a phase II trial was to quickly screen out clinically ineffective agents while minimizing the number of patients exposed to them. Desirable properties of an endpoint in this setting are that it can be evaluated in a short time, it is suggestive of clinical benefit, and it is minimally impacted by patient and disease characteristics so that the mix of patients in the trial would have little influence on the trial results. The endpoint most commonly used that meets these criteria is tumor shrinkage. Early designs were meant to minimize the number of patients exposed to the tested drug, since it was likely to be ineffective and toxic. A drug with a tumor response rate less than 20% was felt to be not promising. The Gehan design¹ was the first used. It is a 2-stage single-arm design in which an initial cohort of 14 patients is accrued. If no responses are observed, the drug is declared ineffective with 95% confidence the tumor response rate is less than 20%. If one or more responses are seen, an additional 11–16 patients are accrued to yield a response rate estimate for which the margin of error is at most 0.10 for the 2-sided 95% CI.

The Gehan design provides limited guidance for determining whether an observed response rate is clinically meaningful and does not provide information regarding the probabilities of type I and type II errors. As more effective oncology drugs were becoming available, a higher standard of evidence for determining the potential for clinical benefit became important. Fleming² proposed a 2-stage design that requires the specification of the smallest response rate that would be considered promising and the largest response rate that would be deemed not to be promising with specified type I and type II error probabilities. Stopping boundaries are developed for the first stage that recommend stopping if the results are drastically positive or negative. If the boundaries are not crossed at the first stage, additional patients are accrued and decision rules are applied to determine whether the drug has potential clinical benefit or not. Simon³ optimized this design with the modification that the trial would stop after the first stage only if the results are deemed dramatically negative. If the stage 1 results are promising, the additional stage 2 patients yield a more precise estimate of the response rate. This design minimizes the average sample size under the null hypothesis that the response rate is not promising and achieves the desired type I and type II error probabilities. The Simon design, or some variation, remains the standard for single-arm phase II trials.^4–6

Recently the increased availability of combination and molecularly targeted therapies has challenged the use of a single-arm phase II trial design. The concern is the reliability of a historical control value. In combination therapies, presumably one or both treatments are effective to some degree and a precise estimate of tumor response rate for each monotherapy is not available. It is likely that molecularly targeted therapies exert disease control through mechanisms other than tumor shrinkage, and so tumor response is not an adequate endpoint. If progression-free survival (PFS) is used, historical estimates are unreliable because this endpoint is influenced by disease and patient characteristics, and the relatively small cohort of patients in the new trial may differ significantly from the historical controls (if available). Furthermore, there may not be a historical control at all if patients are selected on the basis of having the molecular target. The presence of the molecular target may be prognostic and when the molecular target status is not known in the historical control cohort, a reliable historical control value cannot be obtained.

Unreliable historical controls have given rise to randomized phase II trial designs. These trials generally randomize patients to standard treatment (control arm) and one or more experimental arms. A formal comparison is made between the treatment arm(s) and the control arm. There are different variations of a randomized phase II trial, which include the seamless phase II/III trials,^7,8 randomized discontinuation trials,⁹ and Bayesian outcome-adaptive randomization trials.^10,11 Seamless phase II/III designs are being used more often because of the efficiencies gained in terms of protocol development and site contracting. Essentially, the protocol suspends accrual at the completion of the phase II portion and if the phase II trial is positive, it is reopened for phase III accrual. Patients in the phase II portion can be used as part of the phase III portion. Hence there is only the need to develop and get approval for a single protocol, and site contracting needs to be done only once. Another design of recent interest uses Bayesian outcome-adaptive randomization. In this design, the randomization scheme is re-weighted after each patient outcome is observed to more favor the arm with the better outcomes. Hence the next patient randomized has a higher chance of being randomized to the arm that has the best outcomes at that point. This trial design minimizes the number of patients who receive ineffective treatment (if one treatment is better than the others); however, it generally requires larger sample sizes than randomization schemes that have equal probabilities of randomizing among the arms. Finally, another design of interest is the phase II screening design.¹² These often do not have a control arm but rather compare multiple experimental treatments to select the one that has the best outcomes to move forward into a phase III trial.

Randomized phase II trials often use an intermediate endpoint that differs from the endpoint for a definitive phase III trial. For example, the phase II trial endpoint is PFS and the subsequent phase III endpoint is overall survival (OS). Randomized phase II trials sometimes also measure the PFS rate at a specified time point, such as 12 months. This aids in ensuring that the final analysis can occur soon after the last patient enrolled has been followed for the necessary time (eg, 12 months for PFS rate at 12 months) rather than having to wait until a specified number of events has been observed, but this has very modest impact on sample size. Randomized phase II trials generally have a large type I error (eg, one-sided 0.10) requiring a subsequent phase III trial with a more definitive type I error rate (eg, 2-sided 0.05). Allowing larger type I errors and increasing the type II error (reducing power) reduces the sample sizes needed for a randomized phase II trial. This tends to make randomized phase II trials feasible in terms of sample size requirements. In summary, the intent of the randomized phase II is to determine whether a treatment has potential for clinical benefit in a timely manner, that is, to inform a go/no-go decision, and not to provide definitive evidence.

Overall, there are advantages and disadvantages associated with single-arm phase II trials and randomized phase II trials. The advantage of using one design rather than the other depends upon several factors. The remainder of this paper illustrates and discusses the advantages and disadvantages of these designs within the context of neuro-oncology trials.

The Argument for Prioritizing Randomized Phase II Designs for Newly Diagnosed Glioblastoma

The specific value of randomization in phase II is linked to the endpoint of the trial in question, which in turn is dependent on the overall goals of the study. For this discussion, we will assume that the purpose of phase II is to elucidate some measure of biological activity, or “signal,” and to provide sufficient data to adequately inform good phase III go/no-go decision making. An alternative purpose of a phase II trial might be to focus solely on the “signal finding” aspects, but then we would still be left with designing another trial to make decisions about whether to move to phase III. All therapeutic development is associated with risk, but oncology drug development is more likely to fail in the later stages of development, and failures in phase III are common.¹³ Go/no-go decision making would be improved if phase II results could reliably estimate the probability of phase III success.

Phase III trials in neuro-oncology typically use OS as an endpoint, so it is critical that the effects on the endpoints chosen in phase II have some ability to predict therapeutic effects on OS. For clinical trials evaluating novel therapies to treat GBM, several endpoints have demonstrated potential for false signals. Overall response rate has proven to have a poor association with OS¹⁴ and effects on PFS correlate strongly with OS effects for temozolomide,^14,15 but this association does not hold for bevacizumab and might be expected to vary with immunotherapy.¹⁶ Additionally, for a disease with no proven efficacious therapies and short survival time in the post-progression phase, it is questionable whether PFS provides meaningful benefit over OS as a trial endpoint in GBM.¹⁷ Regardless of the endpoint chosen, randomization has utility in separating therapeutic signal from confounders, and in the case of both PFS and OS, randomization is crucial.

The first published randomized controlled trial was designed by Sir Austin Bradford Hill for the Medical Research Council to determine the value of streptomycin in treating pulmonary tuberculosis.¹⁸ Randomization had been strongly advocated in controlled experimentation by R. A. Fisher as a means to accurately estimate sampling error and legitimize significance testing.¹⁹ Hill additionally argued that randomization was important in clinical trials to control for the potential for selection bias resulting in observed outcomes that were attributable to factors other than the experimental intervention.¹⁹ Endpoints with significant natural variability and many potential explanatory variables associated with that variability are more prone to such bias. For example, OS varies substantially among patients and may be attributable to known prognostic factors such as age and performance status in addition to potential unmeasured confounders. Alternatively, overall response rate may be more directly attributable to therapy with fewer alternative explanatory variables. For any endpoint, randomization is used to control for such confounders and improve phase II trial design,²⁰ but it is particularly important for endpoints such as PFS and OS. In addition to selection bias, comparison to historical controls may be prone to false positive results by ignoring the variability in the historical control and not accounting for patient temporal drift.²¹ Comparison of single-arm phase II results to historical data can therefore lead to an overestimation of therapeutic effect and result in poor phase III go/no-go decision making and late stage failures.

Maitland et al²² showed that the overall predictive probability of phase II “success” in combination chemotherapy trials is extremely low and that phase II trials were more likely to claim success if they were not randomized. Simulation studies have demonstrated that while single-arm and randomized studies unsurprisingly are comparable as long as there is a strong historical control for a given endpoint,²³ the addition of selection bias or patient temporal drift can result in significantly higher false positive rates than randomized studies.²³ Patient selection and temporal drift in real world single-arm studies are generally concerns in clinical trials and have been demonstrated in neuro-oncology, specifically. Grossman et al²¹ published the results of 3 separate single-arm trials with diverse mechanisms of action conducted through the New Approaches to Brain Tumor Therapy (NABTT) in comparison with historical data from both NABTT and the European Organisation for Research and Treatment of Cancer (EORTC)/National Cancer Institute of Canada (NCIC) study that defined standard of care. Even though known prognostic factors were similar or of slightly higher risk in the single-arm trials, all 3 trials showed substantially better OS compared with the EORTC/NCIC study but notably also compared with the more internally standardized historical data from NABTT.²¹ The likelihood of having 3 effective drugs in a disease with so few historical successes is low and is more easily explained by the aforementioned problems with historical controls, even when attempts are made to control for the known issues. One of the drugs included in this analysis, cilengitide, had another single-arm phase II trial that was interpreted as promising compared with historical controls.²⁴ Unfortunately, the optimism was not confirmed in CENTRIC, the follow-up randomized phase III study.²⁵ Similarly, the ACT IV trial²⁶ of rindopepimut in addition to standard chemoradiotherapy for newly diagnosed GBM failed to validate the excitement generated from 3 uncontrolled phase II studies.^27–29

The most common criticism for incorporating randomization into phase II trials is based on efficiency. To add randomization requires the addition of a control arm and the variability in outcome associated with that arm, leading to a substantial increase in total number of patients required for the trial. This argument is even more compelling in GBM, where randomization to a standard of care with such poor outcomes is undesirable. Reliable data in the phase II setting prevent additional patients from becoming exposed to ineffective therapies in phase III, however, and there are mechanisms to mitigate the increased patient resources required for randomization. Unequal randomization strategies, such as 2:1, have been employed to be more attractive to patients, but anything without equal numbers of control patients to a given experimental arm results in overall efficiency loss in the trial. Randomized, noncomparative studies ultimately still rely on historical control estimates. Another solution is to have control arms that are common to multiple experimental therapies in platform trials under master protocols. When multiple arms are used in a single clinical trial infrastructure, response adaptive randomization can be added to more efficiently allocate patients to successful arms. Even R. A. Fisher is reported to have suggested that randomization proportions may need to be dynamically altered based on accumulating results in medical trials.¹⁹ Such a design is most notably displayed in practice in the ongoing I-SPY 2 trial for neoadjuvant systemic therapy in breast cancer.³⁰ Translating such elements to GBM³¹ could provide efficiencies for phase II trials based on OS while maintaining 1:1 comparison between promising arms and control.³² The National Cancer Institute’s Brain Malignancy Steering Committee Clinical Trials Planning Workshop included these elements in published recommendations³³ and they are now being included in the recently opened INdividualized Screening trial of Innovative Glioblastoma Therapy (INSIGhT; NCT02977780) and the GBM Adaptive Global Learning Environment (GBM AGILE) trial, currently in development.

In reality, the standard paradigm of phase I → phase II → phase III is too simplistic to describe actual trial goals. For early phase II trials where a change in an imaging or pharmacodynamic biomarker is anticipated based on mechanism of action, a single-arm study may be appropriate to investigate biological activity and determine whether further study is warranted. These signals may also be elucidated from phase I studies, however, so the lines between traditional phases are increasingly blurred. Ultimately, decisions need to be made as to whether an experimental therapy has enough potential to improve OS to warrant the commitment of large patient and financial resources in pivotal studies. For all of the reasons mentioned above, randomization is critical in generating reliable data to make those decisions, particularly for clinical trials in GBM.

The Argument for Prioritizing Single-Arm Phase II Trials for Newly Diagnosed Glioblastoma

Newly diagnosed GBM is a devastating disease. Long-term survival approaches zero and the average life expectancy is 14–16 months in patients who are well enough to join clinical trials^34,35 and less than 12 months in the general population.³⁶ Despite 30 years of research and the accrual of over 30000 patients to high grade astrocytoma trials,³⁷ only 2 drugs (carmustine wafers and temozolomide) have been approved by the FDA for patients with newly diagnosed GBM. Unfortunately, these extend median survival by less than 3 months^15,38 and neither offers a potential for cure. The major barriers to substantial progress include (i) an inability to completely resect this cancer, (ii) the sensitivity of the normal brain to radiation, (iii) failure of the vast majority of systemically administered chemotherapy to penetrate the blood–brain barrier, and (iv) the relative unresponsiveness of high grade astrocytomas to antineoplastic therapies.

This lack of substantial progress in the treatment of newly diagnosed GBMs highlights the urgent need to screen a wide variety of novel approaches for preliminary evidence of activity using phase II trials. This is more challenging than one might expect in this patient population. First, the number of patients eligible and willing to join these trials is limited. GBM is a relatively uncommon malignancy, with approximately 12000 new cases occurring each year in the USA.³⁶ Over 50% of patients with this disease are over 65 years of age. Nationally, only 3% of all cancer patients enroll in clinical trials, and the elderly participate at a substantially lower rate.³⁹ Unique factors that can limit participation in clinical trials for newly diagnosed GBM include a history of seizures that may affect a patient’s ability to drive to trial-related tests or treatments and neurological deficits that may interfere with the ability to provide informed consent.

A second challenge relates to how difficult it is to assess the efficacy of trial interventions in patients with newly diagnosed GBM. Judging response by physical exam is fraught with error as neurological deficits may not recover even as tumor regresses. Similarly, neurological deficits may improve with therapies that treat peritumoral edema or worsen with interventions that cause brain edema, regardless of the status of the underlying tumor. Neuro-imaging endpoints are also often unreliable, as contrast enhancement in CT and MRI provides direct information on the extent of blood–brain barrier integrity rather than the size of the tumor. As a result, these scans typically look better after treatment with glucocorticoids or vascular endothelial growth factor–targeted therapies and worse following radiation independent of the status of the tumor. The discordance between neuro-imaging and outcome has recently been highlighted in phase III trials of bevacizumab which prolonged stability on MRI scans but had no impact on survival.^34,40 These observations complicate the use of surrogate response markers, such as PFS, leaving OS as the best endpoint for phase II studies in this population.

The ultimate goal of phase II trials in newly diagnosed GBMs is to rapidly screen new agents and approaches for an early efficacy signal using as few patients as necessary. In an ideal world with few constraints on available patients and resources, randomized phase II trials would be the preferred approach, as they should significantly reduce the false positive rate by reducing bias and increasing confidence in the assessment of drug effect.^41,42 However, a recent review of phase II oncology clinical trials that progressed to phase III studies failed to demonstrate a significant difference between single-arm and randomized phase II designs in predicting the eventual phase III outcome.⁴³ More importantly, given the long list of negative trials in patients with GBM, the primary philosophic thrust driving the design of phase II studies might favor designs focused on identifying ineffective therapies for early exclusion (single-arm phase II) rather than designs assuming that each new drug might be efficacious (randomized phase II). Furthermore, as patients for these trials are limited, other features of phase II trial designs figure prominently. Randomized phase II trials require significantly higher patient numbers per study due to the inclusion of an internal control arm which receives standard therapy. This design also increases the time to study completion and the costs involved, thereby reducing the number of novel agents that can be studied. Finally, many physicians and their patients who are willing and eligible to participate in new drug studies have a strong preference for nonrandomized studies.⁴⁴

Single-arm phase II studies also present special challenges. Historical controls with similar eligibility criteria are required for the results of these studies to be interpretable. Fortunately, the newly diagnosed GBM literature currently contains contemporary randomized phase III studies with well-defined control arms which can provide relevant outcome data.^{15,25,34,35,40} However, when a matching historical control group is not available, a randomized phase II design should be employed. For example, when molecularly defined subgroups are selected for study, the natural histories of these distinct subgroups are usually unknown, and thus a concurrent control arm would be necessary to accurately determine the impact of any intervention. In addition, the most robust and reliable endpoints should be used when conducting a single-arm study to minimize confounding variables. In newly diagnosed GBM trials the most reliable endpoint is OS. An appropriate concern in using historical controls for GBM trials is that patient survival can improve over time as clinicians become more inclined to do multiple surgeries, re-treat patients with radiation, and offer better supportive care.²¹ For this reason, contemporary historical controls are favored.

Given the discouraging outcomes of clinical studies in adults with GBM over the past several decades, it is relatively safe to presume that the vast majority of appropriately designed single-arm phase II studies evaluating novel therapies will fail to demonstrate a noteworthy survival signal and thus would not be candidates for further development. An encouraging result from a single-arm study should be considered only as a preliminary signal of efficacy and should be followed by a confirmatory randomized phase II study with or without provisions to transition to a phase III design.⁴²

Conclusions

Phase II trials are critically important in the development of novel therapies for patients with newly diagnosed GBMs. These trials represent the initial evaluation of activity in this disease and are specifically designed to triage novel compounds and approaches into those that do and do not deserve further study. While it often appears that there is a conflict between randomized and single-arm phase II trial designs, in reality each has its place in furthering effective drug development. Since the vast majority of experimental drugs tested in patients with newly diagnosed GBM have been clinically ineffective, designing small single-arm phase II studies to eliminate ineffective therapies early is reasonable. However, larger randomized phase II trials are important to reduce confounders and false positives. Investigators should carefully consider the trial endpoint, the availability of appropriate controls, and how trial results will be used to inform further development of the experimental therapy when choosing a design. Ultimately, single-arm and randomized phase II trials (as well as adaptive and integrated designs) provide useful tools for evaluating the efficacy of novel therapeutics in newly diagnosed glioblastoma when applied in an appropriate fashion.

Funding

This manuscript did not rely on any funding.

Conflict of interest statement. None for any author.