Factorial clinical trials: a new approach to phase II neuro-oncology studies

Glioblastoma (GBM) is the most common and aggressive primary brain tumor in adults.¹ Standard initial treatment consists of maximal surgical resection followed by concurrent radiotherapy and temozolomide chemotherapy and then at least six months of adjuvant temozolomide (in the absence of disease progression or unacceptable toxicity).² However, nearly all tumors eventually recur, ultimately leading to death within 1–2 years from diagnosis.² Innumerable clinical trials have tested new agents, each with promising pre-clinical data. Some demonstrated encouraging phase II efficacy results, but nearly all failed when tested in a randomized phase III context. Recent examples include the vascular endothelial growth factor inhibitor bevacizumab^3,4 and the integrin inhibitor cilengitide⁵ for newly diagnosed GBM. New approaches to screen drugs for efficacy rapidly in the phase II setting are needed.

Simultaneously, the need to combine agents is pressing because the number and types of anticancer therapies, including signal transduction inhibitors, antiangiogenic agents, and more recently immunotherapies, has exploded in recent years. Many have proven activity in and FDA approval for other cancers. However, obvious, validated, and drugable driver mutations in GBM are elusive, and it is also possible, if not likely, that many GBMs are driven by multiple molecular abnormalities requiring multiple simultaneous approaches.

A factorial approach is one method to test multiple drugs simultaneously and efficiently. Dr. Penas-Prado and her team are to be congratulated on their diligence for designing such a phase II study.⁶ They enrolled patients with “newly diagnosed” GBM (although after concurrent chemoradiotherapy but without disease progression, rather than treatment naïve patients) and treated them with three experimental therapies, isotretinoin (a pro-differentiating agent), celecoxib (a COX-2 inhibitor), and thalidomide (a purported antiangiogenic), along with dose-dense temozolomide. These agents were tested in 8 different arms under one IRB-approved randomized clinical trial: arm 1, dose-dense temozolomide alone; arms 2–4, doublets of dose-dense temozolomide with each of the other agents; arms 5–7, triplets of dose-dense temozolomide plus two of the three other agents; arm 8, the quadruplet of all four agents.⁶ The primary endpoint was to determine the benefit, measured by progression-free survival (PFS) from each of the experimental drugs. For example, to determine efficacy of isotertinoin, PFS of patients treated with istotretinoin (pooling all isotretinoin containing arms) was compared against PFS of patients not treated with istretinoin (pooling all non-isotretinoin containing arms). The same analyses were conducted for the other agents. By design, the study was not powered for arm-to-arm comparisons, as each arm was to accrue only 20 evaluable patients.

Factorial design studies were developed in the late 1800s and early 1900s and have been widely used in other contexts, typically those using highly controlled conditions with low variability, such as agriculture or machinery. Attributed mainly to the statistician Sir Ronald Aylmer Fisher, they allow multiple questions to be asked and answered simultaneously as part of the same experiment. Fisher objected to the notion that multiple experiments were impossible to conduct simultaneously, expressing his view that “no aphorism is more frequently repeated in connection with field trials, than that we must ask Nature … one question, at a time. The writer is convinced that this view is wholly mistaken.”⁷

In clinical research, factorial designs have been successfully employed in prevention studies such as the 2 × 2 factorial design Physician's Health Study (aspirin, beta-carotene, both, or none) for reducing cardiovascular or cancer incidence and the 2 × 2 factorial design of alpha-tocopherol, beta-carotene, both or none for lung cancer prevention.⁸ The track record of factorial design in therapeutic trials in oncology is mixed.⁹

A major advantage of the factorial design is symmetry; i.e. each individual experimental drug is given to 50% the subjects, so each research subject's data contributes to many treatment comparisons. For example, although there were only 20 patients per arm, approximately 80 patients received isotretinoin and 80 did not in the trial under discussion.⁶ More importantly, the ability of the factorial design to simultaneously assess 3 different experimental drugs in one clinical trial certainly allows fewer subjects to be enrolled when compared to 3 conventional trials of 1-experimental-drug-a-time. This potential gain in efficiency and reduced sample size for phase II trials in neuro-oncology is welcomed, where the main goal is screening of drugs with potential activity for larger confirmatory phase III trials.

However, there are several potential limitations to the factorial design. One obvious issue is the multiple testing problem, whereby the risk of accepting an intervention as effective is artificially elevated. For example, in the trial by Penas-Prado et al,⁶ there were at least 3 pre-planned comparisons (isotretinoin versus no isotretinoin, celecoxib versus no celecoxib, and thalidomide versus no thalidomide), and the alpha error was kept at 0.05; therefore, the risk of type I (false-positive) errors is higher in the factorial design compared to conventional 2-arm/1-experimental drug designs. This did not manifest as a problem in the trial by Penas-Prados et al because all pre-planed comparisons were negative. Nevertheless, the suggestion that both progression-free survival (PFS) and overall survival (OS) were shorter following arm-to-arm comparison of isotretinoin plus dose-dense temozolomide versus dose-dense temozolomide alone (or put differently that dose-dense temozolomide was superior) could have falsely resulted from the multiple testing problem. It also could have resulted from heterogeneity of the patient population, with different prognostic or predictive factors, a limitation acknowledged by the authors. This is a reason why 2 × 2 × 2 factorial design cannot accurately compare each of the 8 arms separately. Notably, there was no statistically significant difference in PFS when comparing results from all patients who received isotretinoin versus all who did not, perhaps because the heterogeneity was less of an issue with a larger sample size of approximately 160 patients overall (80 vs. 80) rather than the small sample size in arm-to-arm comparisons (20 vs. 20).

There are other potential limitations to factorial approaches. First, intra-patient or inter-patient changes in drug dosing would inject excess variability, limiting validity of comparisons. Fortunately, Gilbert and colleagues previously established the doses through a phase I study,¹⁰ and all arms in the phase II study received the full dose of each drug.⁶ Second, drugs of very different class of action, as used by Panos-Prado et al are the most appropriate for factorial design trials. Third, if synergistic interactions between or among treatments are hypothesized, then the approach of testing one factor at a time by combining arms, as is done in factorial design trials, may not be the most appropriate or efficient.

In the trial by Penas-Prado et al, unfortunately, none of the 3 experimental drugs demonstrated efficacy.⁶ While disappointing, such results are nonetheless important to report. The authors demonstrated in well-powered analyses that none of the 3 drugs tested (thalidomide, isotretinoin or celecoxib) deserves to move forward to larger confirmatory studies as the phase II results clearly showed no signal of potential efficacy. They demonstrated both the feasibility of a factorial approach, and its efficiency. In addition, writing and submitting a manuscript associated with a negative trial is often a joyless endeavor, especially when interest has waned and both the investigators and the field have focused on other more “exciting” studies. Even if submitted, reviewers and journal editors may be biased against publication of negative results: positive trials are accepted to journals more easily than negative ones.¹¹ An average of 51 months following trial completion, 32% of 635 NIH-funded clinical trials remained unpublished in any peer-reviewed journal.¹² This trend appears to be worsening over time, as only 7% of trials funded by the NIH in 1979 remained unpublished 10 years later.¹³ However, the NCI has made that clear publication of negative trials is as important as positive ones, and new policies will likely mandate publication.¹⁴ We learn from our missteps as much if not more than from our successes. This is particularly important in neuro-oncology, where trials based on solid science and pre-clinical data all too frequently fail to produce clinical benefit. Unfortunately this trial was no exception.

Accordingly, we fight on, seeking different drugs for new targets, or better drugs for familiar ones.

Funding

A.B.L. was supported in part by P30 CA013696 and UG1CA189960