More Randomization in Phase II Trials: Necessary but not Sufficient

Historically, phase II trials in oncology were generally single armed, constructed to distinguish between a tumor response rate felt to indicate a lack of promise (often 5%) and a rate that would indicate potential benefit (often 20%), with a one-sided type I error rate of 5%–10% and a type II error rate of 10%–20% (1). The dominant use of this design was based on the premise that an agent that could not produce a tumor response rate of 20% was not likely to produce a clinically meaningful overall survival (OS) or progression-free survival (PFS) benefit in subsequent phase III testing. Recent trends in oncology drug development have challenged this paradigm. Many phase II trials are now designed to assess the promise of a molecularly targeted agent, given either alone or in combination with another regimen. In many cases, these agents are not anticipated to produce or improve tumor response rates; rather, the desired outcome from their use is improved PFS or OS through means other than direct cell killing as evidenced by tumor shrinkage (2). In general, PFS is the preferred endpoint for such phase II trials. PFS is statistically more efficient than OS because the time to achieve the endpoint of PFS is substantially shorter, and the treatment effect is not diluted by salvage treatment. However, in a situation with no effective salvage therapy and/or a disease with concerns regarding the timing of progression assessment, OS could be chosen as the endpoint. Such trials can be single-arm studies, compared with historical controls, or can be randomized.

The review by Sharma et al (3) in this issue of the Journal is a welcome addition to the growing chorus in favor of increased randomization in phase II trials for agents with little likelihood for single-agent tumor regression and for which endpoints such as PFS are used. This promotion of randomization is already having dramatic effect. Current records of the Cancer Therapy Evaluation Program of the National Cancer Institute (NCI) reveal that only 1.5% (68/4437) of the completed NCI-sponsored phase II studies were randomized. In contrast, 28% (69/243) of the currently active phase II studies are randomized, and of the trials activated after December 31, 2009, 37% are randomized. A primary reason for this increase is the appreciation, in the trial design and review process, that even a modest upward drift in the PFS of the study population compared with historical controls, which is independent of the effect of the new agent being tested, can inflate the type I error rate approximately threefold (4). For example, a drift from 50% to 55% in the control 4-month PFS rate, when not accounted for, will increase the type I error of a single-arm Simon optimal trial (5) targeting a 70% 4-month PFS from 0.10 to 0.26. Coupled with this is the realization that such an upward drift over time is relatively likely for PFS as the standard of care improves (6).

It is widely accepted that a substantial portion of phase II trials will still be appropriately single arm (1,6–8). This includes trials of agents for which tumor regression is anticipated based on mechanism of action, as well as early phase II monotherapy trials to establish a tumor response signal of biological efficacy. Additionally, monotherapy and combination trials with PFS endpoints in diseases with no effective standard therapy and established stable historical controls (eg, recurrent glioblastoma) can be justified. For OS, an historical database for melanoma has proven useful for designing single-arm studies (9). In some situations, adjustment for observed differences in the distribution of known prognostic factors between the historical database and the observed single arm study can reduce potential bias and strengthen inferences. Examples of adjustment strategies include both regression models and probability reweighting (10).

Importantly, expanding the use of randomization to all phase II situations in which it is appropriate will not by itself maximize the positive predictive value of phase II trials (the probability of a positive phase II trial yielding an agent or combination that is effective in subsequent definitive phase III trials). This value is dependent not only on the type I error rate of the phase II trial but also on the balance between the true- and false-positive rates of the phase II trials for the population of interest, as well as the degree to which the phase II endpoints predict the ultimate phase III endpoints. For example, if the type I and type II error bounds are both .10, then the positive predictive value of a phase II trial will vary between 32% and 61% in the setting in which first, the collection of agents and combinations tested is effective, with probability varying between 5% and 15%, according to the phase II endpoint, and second, the phase II endpoint is a perfect surrogate for the phase III endpoint. Because the second stipulation is never the case, the positive predictive value may be substantially less.

There are four potential approaches to maximizing the effectiveness of phase II trials as predictors for phase III success:

1. The pool of agents and combinations going into phase II testing can be enriched for truly active agents. Enrichment may be possible through the increased use of pharmacodynamic assays in phase I and phase 0 testing (11), allowing for go/no-go decisions before phase II testing. Additional single-arm clinical data (potentially collected at phase I or phase II) may be helpful for screening for agents before undertaking randomization.

2. The subpopulations in which agents and combinations are potentially effective can be better identified so that phase II testing can be limited to such subpopulations. This may be done by increased development and use of pharmacodynamic assays to better characterize the agents (11) and increased development and use of biomarkers to better identify correspondingly sensitive subpopulations of patients (7,8).

3. Phase II endpoints that capture and predict a substantial percentage of the treatment effect reflected in the ultimate phase III endpoints can be identified, established, and used (7,12). Such endpoints, including new imaging endpoints, may vary by class of agent and by disease (13,14).

4. Even if the approaches listed above are only modestly successful in enriching the pool of phase II agents and combinations, so that they are effective, with probability varying between 20% and 40%, according to the phase II endpoint, the positive predictive value of phase II trials (to reflect true efficacy according to the phase II endpoints) could be increased to between 69% and 86%. How well these phase II trials would then predict phase III efficacy would depend upon the proportion of the phase III treatment effect captured by the phase II endpoint. However, in situations in which the above approaches are not so successful in enriching the pool of phase II agents and combinations, conducting phase II trials at the significance level of .05 (rather than the .10) should be considered. In this way, even if the agents and combinations are effective, with probability varying between 10% and 20%, according to the phase II endpoint, the positive predictive value of phase II trials to reflect true efficacy according to the phase II endpoints would vary between 67% and 82%.

A longer-term issue is whether the conduct of randomized phase II trials designed to evaluate PFS endpoints for agents that do not induce tumor regression as single agents represents a productive strategy. It is too early for conclusions, but there is emerging evidence to suggest that future oncologists will view this strategy as having identified only marginally to modestly effective agents. To date, many of the agents requiring phase II randomized trial designs because of their lack of tumor-regressing activity for the disease being studied are angiogenesis inhibitors (eg, agents targeting vascular endothelial growth factor [VEGF/VEGF receptor {R} 2] signaling). This class of agents has been remarkably effective for renal cell carcinoma, and multiple agents in the class have achieved regulatory approval for this indication (15–18). However, this is a tumor type for which many agents targeting VEGFR2 have single-agent tumor-regressing activity (16,19,20). Outside of cancers such as renal cell carcinoma and thyroid cancer, for which substantial single-agent response rates are observed (16,19–22), the track record for VEGF pathway inhibitors has primarily been one of the failed phase III trials or the phase III trials that have resulted in only modest prolongations in PFS and in some cases small increases in OS (23–31), although there are exceptions (32). This limited return on investment is in marked contrast to the improvements in outcome that have been observed for agents prioritized on the basis of their ability to induce substantial rates of regression in classic phase II studies in patients with recurrent/refractory disease. The strategy of prioritizing agents with robust single-agent activity has led to substantial improvements in survival, as demonstrated in phase III trials in a number of settings, such as arsenic trioxide for acute promyelocytic leukemia (33), tretinoin for acute promyelocytic leukemia (34,35), imatinib for chronic myeloid leukemia and Ph⁺ acute lymphoblastic leukemia (36,37), rituximab for various types of B-cell non-Hodgkin lymphoma (38,39), trastuzumab for human epidermal growth factor receptor 2–positive breast cancer (40), and lenalidomide for multiple myeloma (41). A separate set of design and prioritization issues that are beyond the scope of this editorial apply to agents lacking single-agent tumor-regressing activity for which there is evidence of synergistic or synthetically lethal interactions when used in combination with other agents.

In summary, Sharma et al. (3) provide a rationale for selecting randomized phase II designs over single-arm designs when evaluating agents that lack single-agent tumor-regressing activity. An open question is whether such agents will be able to provide the level of improvement in survival that patients and their physicians seek.

Funding

This work was supported, in part, by National Cancer Institute (R01 CA090998 to M.L.).