Trial Design Challenges and Approaches for Precision Oncology in Rare Tumors: Experiences of the Children’s Oncology Group

Abstract

In the United States, cancer remains the leading cause of disease-related death in children. Although survival from any pediatric cancer has improved dramatically during past decades, a number of cancers continue to yield dismal prognoses, which has motivated the continued study of novel therapeutic strategies. Furthermore, even patients cured of pediatric cancer often experience severe adverse effects of treatment and other long-term health implications, such as cardiotoxicity or loss of fertility. For these patients, improved risk stratification to identify those who could safely receive alternate or less-intensive therapy without affecting prognosis is a key objective. Fortunately, pediatric cancers are rare overall, but even among patients with the same narrow cancer type, there is often broad heterogeneity in terms of prognosis, molecular features or pathology, current treatment strategies, and scientific objectives. As a result, the design of clinical trials in the pediatric cancer setting is challenged by a number of practical issues that must be addressed to ensure trial feasibility for this vulnerable group of patients. In this review, we discuss some of the unique trial design considerations often encountered in any rare tumor setting through the lens of our experiences as faculty statisticians for the Children’s Oncology Group, the largest organization in the world dedicated exclusively to pediatric cancer research and clinical trials. These topics include risk stratification within individual trials, relaxation of trial operating characteristics and parameters, use of historical controls, and address of noninferiority-type objectives in small cohorts. We review each in terms of practical motivation, present challenges, and potential solutions described in the literature and implemented in selected example trials from the Children’s Oncology Group.

INTRODUCTION

Overview of Pediatric Cancer and Current Directions

In the United States, cancer remains the leading cause of disease-related death in children.^1,2 Although the prognosis for most pediatric patients has improved dramatically during the past several decades to achieve an overall 5-year survival rate of 84%, pediatric cancer truly comprises many heterogenous diseases with a great deal of variability in expected survival outcomes and current scientific objectives.^3,4 Among the childhood cancer types with the best prognosis (thanks in large part to decades of therapeutic intensification trials for improved survival), achievement of an incremental improvement upon current outcomes remains a key objective of most pediatric trials alongside objectives and novel treatment strategies intended to minimize short-term treatment-related toxicities and improve longer-term considerations, such as fertility, cardiac health, and quality of life.^5-9 For other childhood cancers, such as acute myelogenous leukemia, high-risk neuroblastoma, and brainstem glioma, progress has been more limited, with 50% or more patients eventually succumbing to their disease. In those diseases, the study of novel therapeutic strategies to improve overall survival (OS) remains the highest priority.^10,11

Role of the Children’s Oncology Group

The Children’s Oncology Group (COG), a member of the National Clinical Trials Network supported by the National Cancer Institute, is the world’s largest organization devoted exclusively to childhood and adolescent cancer research. The current organizational structure of COG was established in the year 2000 as the result of a voluntary merger of its four predecessor groups: the Children’s Cancer Study Group (CCG), Pediatric Oncology Group (POG), Intergroup Rhabdomyosarcoma Study Group, and National Wilms Tumor Study Group. More than 90% of the 14,000 children and adolescents newly diagnosed with cancer each year in the United States are cared for at COG member institutions, with more than half of these patients historically enrolling in COG therapeutic clinical trials. At the end of 2017, more than 25,000 children were being treated either in COG trials or in active follow-up in COG studies at more than 220 leading children’s hospitals, universities, and cancer centers across the United States, Canada, Australia, and New Zealand.

CONTEXT

• Key Objective

• Clinical trials in pediatric cancer and other rare diseases often include unique statistical features or design challenges.

• Knowledge Generated

• In the pediatric setting, common design components include use of prognostic risk stratification and use of data from historical controls. Statistical challenges include maintaining power or addressing noninferiority-type hypotheses when only a limited population of patients is available to enroll.

• Relevance

• We provide an overview of these features and challenges and describe existing approaches in the literature as well as those used in example trials from the Children’s Oncology Group.

Pediatric Cancer as a Rare Disease With Trial Planning Implications

Fortunately, even the most common pediatric cancers occur only rarely in the general population, representing only 1% of new cancer diagnoses in the United States.¹² Also fortunate is the current availability of effective therapies for most of these patients. However, both the rarity of pediatric cancers and their generally favorable outcomes present unique challenges to clinical trial design, where improving outcomes and lessening the burden of treatment remain critically important.

In some pediatric malignancies, it is not feasible to conduct a randomized phase III clinical trial to detect a modest, yet clinically relevant improvement in outcome with high statistical power and a low type I error rate. The number of patients who would be required to enroll in such a trial during a relatively short period simply does not exist in the general population. Furthermore, with targeted therapies now increasingly regarded as promising treatment strategies for patients whose tumors harbor particular molecular features or mutations, the already small disease populations that currently comprise pediatric cancer are becoming even smaller as a result of additional molecular characterization, thereby exacerbating some of the usual trial design challenges inherent in any rare disease. In addition, for those pediatric cancer subtypes where exceptional prognosis or cure can now reasonably be expected with standard treatment, current and future trials likely will be designed to answer questions such as whether historically toxic treatments can be safely reduced or altered in some way without negatively affecting outcomes. These next-generation questions, although important to answer, generally necessitate noninferiority (NI)-type trial design, which on average, requires a far greater sample size than a traditional phase III randomized trial.

In this article, we highlight several trial design features and challenges that are common to pediatric cancer trials and, in some cases, are broadly applicable to trials for any rare cancer. These include risk stratification as a design component, relaxation of trial operating characteristics to maintain feasibility, alternative approaches to testing NI-type hypotheses, and use of historical controls. For each topic, we provide some brief motivating background; give an overview of ad hoc, formal, and/or novel design solutions described in the literature and commonly used by COG; and highlight applications to our own trials as examples.

RISK STRATIFICATION

Rationale for Risk Stratification in Pediatric Trials

Upfront stratification of patients with the same or similar cancer type into distinct prognostic risk groups is a common practice in pediatric clinical trials. These risk groups might be derived from a number of patient and disease factors, such as age of the patient, size or local stage of the tumor, tumor histology, molecular biomarkers, or other features that have been previously shown to predict patients’ natural course of disease or response to standard treatments. In most COG trials where upfront risk stratification takes place, patients assigned to different strata (eg, low risk, intermediate risk, high risk) will go on to receive (or be randomly assigned to) treatment strategies of increasing intensity, depending on the severity of their initial risk assignment. In some pediatric cancer types, therapeutic de-intensification is the main objective for those known to be at low or very low risk of adverse outcomes, such as disease recurrence or progression.

Challenges for Risk Stratification in Rare Diseases

Although risk stratification offers a logical and feasible approach to studying prognostically different cancer stages or subtypes under a unified protocol, this practice presents some challenges. First, it is common in pediatric cancers for stratum-specific risk classifications to change over time because knowledge of the individual diseases and their corresponding standard treatments continue to evolve. The group of patients considered to be very low risk during one era of treatment trials of a given cancer might be redefined to become more inclusive in the next era, whereas discovery of a new negatively prognostic biomarker present across all risk groups might add a layer of upstaging from lower- to higher-risk groups across the board. Another challenge of risk stratification is that introduction of a new risk factor or threshold to distinguish lower-risk from higher-risk patients might be based on limited patient numbers or incomplete clinical evidence (eg, a biomarker collected for only a subset of patients), which precludes a statistically rigorous evaluation of the risk classification system in question and leaves much to the discretion and instincts of study investigators. Even the methodology originally used to determine that a particular patient or disease feature is sufficiently prognostic to map a patient to a different risk level (and thus treatment assignment), such as whether a patient’s age is above or below some threshold at the time of diagnosis, might be questioned or re-evaluated after more data become available. Modifications, updates, or other inconsistencies between risk classifications from one generation of trials to the next within a single disease type can lead to challenges in defining risk group–specific historical controls when planning the next generation of trials, particularly when available cohorts of reference patients are already small or systematically different in some other way (historical controls will be discussed subsequently in this article). Nonetheless, risk stratification remains central to many pediatric cancers, and thus, the development of optimal approaches for risk stratification in clinical trials remains of critical interest to COG researchers.

Examples of Risk Stratification in COG Trials

Hepatoblastoma.

Hepatoblastoma is one of the most common childhood hepatic tumors, with a population prevalence of 0.5 to 1.5 per 1 million and comprising approximately 1% of all newly diagnosed pediatric cancers.¹³ Because of its relative rarity, however, development of a risk stratification system to inform prognosis, treatment, and design of future clinical trials was both critical and challenging. Initially, several important prognostic factors, including tumor resectability¹⁴ and histology, that are independently predictive of outcome in hepatoblastoma were identified. On the basis of these findings, COG opened the hepatoblastoma studies INT-0098¹⁵ in 1989 and P9645¹⁶ in 1999, using resectability at exploratory surgery and histology information to define a risk stratification framework. In the early 2000s, the radiology-based Pretreatment Extent of Disease (PRETEXT) staging system became a dominant factor in risk stratification because exploratory surgery was no longer routinely supported with the advent of advanced radiographic imaging techniques. At that time, PRETEXT also was already well accepted among other international groups. In 2009, COG opened a new hepatoblastoma study, AHEP0731 (ClinicalTrials.gov identifier: NCT00980460), that incorporated PRETEXT and α-fetoprotein in the risk stratification.

As noted previously, the performance of direct comparisons of patient outcomes from trials or groups that use dissimilar risk classification strategies is challenging, and such was the case in hepatoblastoma. To address this issue, the Childhood Hepatic Tumor International Consortium established a database that consists of 1,605 patients from eight prior studies from COG, the International Childhood Liver Tumors Strategy Group, the German Society for Pediatric Oncology and Haematology, and the Japanese Study Group for Pediatric Liver Tumors.¹⁷ The goal was to develop a common risk stratification system for pediatric hepatic tumors. Multivariable retrospective analyses were performed to create a comprehensive risk stratification algorithm capable of predicting event-free survival (EFS). The identified risk factors include PRETEXT and annotation factor, age, and α-fetoprotein levels. Figure 1 illustrates the details of this new risk stratification tool, which has been implemented in the Pediatric Hepatic Malignancy International Therapeutic Trial (PHITT) with participants from COG, the International Childhood Liver Tumors Strategy Group, and the Japanese Study Group for Pediatric Liver Tumors. This study is investigating treatment reduction for patients with good prognoses and intensification of therapy to improve outcomes for patients with poor prognoses. The biomarker and pathology data that will be collected in PHITT also could potentially provide further refinement to the newly proposed risk stratification.

FIG 1.

Children’s Hepatic Tumor International Collaboration hepatoblastoma stratification. AFP, α-fetoprotein; M, metastatic; PRETEXT, Pretreatment Extent of Disease; VPEFR+, one or more of V (involvement of vena cava or all three hepatic veins, or both), P (involvement of portal bifurcation or both right and left portal veins, or both), E (extrahepatic contiguous tumor extension), F (multifocal liver tumor), or R (tumor rupture at diagnosis) present.

Acute lymphoblastic leukemia.

Acute lymphoblastic leukemia (ALL) is the most common cancer diagnosed in children, with an estimated 3,500 diagnoses per year in persons younger than 20 years of age in the United States.¹⁸ In the current era, overall cure rates for children with newly diagnosed ALL are approaching 85%.¹⁹ Although survival gains during the past four decades are noteworthy, many children predicted to be at low risk of relapse at initial diagnosis ultimately experience treatment failure. Therefore, a goal of COG has been to refine current risk stratification algorithms further to intensify therapy for patients identified to be at a considerable or high risk of relapse while lessening therapy and therefore sparing toxicity for those already highly likely to be cured.

From 2010 until 2018, the COG trial AALL08B1 (ClinicalTrials.gov identifier: NCT01142427) was a classification protocol that provided an entry point for risk stratification and subsequent assignment to therapeutic protocols (or standard of care) for patients with newly diagnosed ALL. Data from legacy studies conducted by the earlier CCG and POG, as well as data from the predecessor COG ALL risk group classification protocol AALL03B1, formed the basis for the refinement of the risk stratification algorithm subsequently used in AALL08B1.

After initial registration and risk stratification in AALL08B1, patients could be deemed eligible for a COG frontline ALL treatment trial as follows. Patients were initially assigned to an induction treatment regimen on basis of age, WBC count, the presence of extramedullary disease, and immunophenotype. Additional studies performed at local and COG reference laboratories at the time of initial diagnosis and at defined time points during induction were used to refine postinduction therapy. Patients with National Cancer Institute standard-risk and high-risk²⁰ B-lymphoblastic leukemia (B-ALL)²¹ were then distributed among the following four risk groups after induction: low risk, average risk, high risk, and very high risk (Table 1). These groups were then treated in separate therapeutic studies.

TABLE 1.

Overview of Risk Classification System for the B-Cell Cohort of Trial AALL08B1

RELAXING TRIAL OPERATING CHARACTERISTICS AND DESIGN PARAMETERS

Pediatric Cancer as a Rare Disease and the Need for Small Trials

Because pediatric cancer is rare in general and rare pediatric cancers are even rarer, slow accrual often plagues clinical trials, even when participation is consistent across the enrolling sites (which is generally the case for COG trials.)^22,23 To maintain reasonable trial duration and overall feasibility, it is often necessary to keep trial sample sizes as low as possible while still maintaining the ability and power to address the scientific objectives of interest.

Improved outcomes that have been obtained in many cancer types in recent decades have compounded the design challenge of pediatric cancer’s low incidence, specifically the limited number of key outcome events (eg, disease recurrence, death) likely to be observed across patients with a reasonable amount of follow-up. Although a welcome “problem” is that children with cancer are living longer than ever before, a lower event rate within a given disease setting directly reduces the statistical power of a clinical trial, unless the sample size is increased (which may or may not be feasible). During the past 50 years, the 5-year survival rate for pediatric cancers has risen dramatically from 10% to 83%, which makes future clinical trials in pediatric cancers more challenging to design.^24,25 Taking standard-risk ALL as an example, the cure rate is 90% or greater for children in the general population.²⁶

Yet another challenge in rare and pediatric cancers is the advent of stratified or precision medicine, with an ever-increasing number of new targeted therapies available for testing within narrowly defined risk-stratified or molecularly defined cohorts. Even among pediatric cancers with the highest incidence rates (eg, ALL, which accounts for approximately 30% of all cancers in children), most trials are conducted within far-smaller cohorts defined by risk stratification systems wherein a given risk stratum has unique objectives and possible treatment assignments, as described in the Risk Stratification section. In such a setting, even a common cancer usually is studied as a series of essentially independent, smaller trials and is therefore subject to many of the same design challenges as rarer cancers.^27,28 When layered upon the general rare cancer issues previously described, precision medicine creates an additional urgency at the design stage to strike a balance between what is statistically rigorous and what is acceptable to support feasibility.

Rationale for Relaxing Operating Characteristics/Design Parameters and Common Approaches

Under current conventions, a typical two-arm randomized phase III clinical trial that tests a superiority hypothesis often uses a two-sided 5% type I error rate (which corresponds to a one-sided 2.5% type I error rate) and power in the range of 80% to 90% to detect a clinically relevant difference in outcome between treatment arms. With a limited patient population from which to recruit because of low incidence, a very long trial would be required to achieve a trial’s full sample size. For example, assuming a cancer with a baseline cure rate of 50% where improvement to 65% cure is desired but only 40 patients per year will likely enroll, a two-arm randomized trial would require the recruitment of 360 patients accrued during 9 years to detect the desired improvement with 80% power and a 5% type I error rate.²³ In pediatric cancer, an accrual rate of 20 to 50 patients per year is common, such as in the setting of risk-stratified relapsed ALL. Although a prolonged trial achieves the typical type I and type II error rates and increases our confidence in the conclusions we obtain from a completed trial, it limits the number of trials we can conduct in the long run and reduces the number of new treatment strategies or agents that we can test. In addition, a treatment can become irrelevant after a long duration, and a long trial can become obsolete before it is completed.^23,29 Therefore, other designs are considered for clinical trials in small populations.³⁰

At COG and in other groups that study rare diseases, a number of approaches commonly are used to minimize the overall sample size required for a trial and are subject to context-specific constraints. These include (among others) targeting a larger effect size and relaxing type I error or power.

Targeting a larger effect size.

When the enrollment in a trial is not sufficient to achieve the sample size that an ideal design would require, one option is to target a larger effect size than the smallest effect that would be clinically relevant (eg, to detect a hazard ratio [HR] of 0.60 [40% risk reduction] rather than a more modest HR of 0.80 [20% risk reduction]). In doing so, the required trial sample size is reduced as one raises the bar of the expected treatment benefit under the alternative hypothesis.^27,31-37 Particularly in low-incidence cancer settings where only one novel therapy can be studied at a time (to avoid splitting the population between competing trials), it can be reasonable to power a trial to detect a large treatment effect so long as such an effect is believed to be feasible given the type of treatment under study and the intended population.³⁴ When the cohort to be studied is so small as to preclude formal hypothesis testing, a primary objective may instead be to enroll as many patients as possible within a given time frame and estimate the outcomes of interest (eg, EFS at 3 years). Although perhaps not conclusive in a strong statistical sense, such a design is often useful to plan subsequent studies and therapeutic strategies.

Relaxing type I error.

Another sample size reduction approach particularly common in phase II and some phase III trials is a relaxation (an increase) of the maximum allowable rate of type I error or the probability of a false-positive result.^{23,27-29,33-35,38,39} A conventional rule of thumb for many clinical studies (not just clinical trials) is to specify a two-sided type I error rate of 5% to control the probability of false-positive results when the null hypothesis (no treatment effect) is true. In setting the type I error rate higher than 5% (eg, 10%, 15%, 20%), a higher risk of false-positive findings is accepted, which means a greater likelihood that a trial will suggest that the experimental treatment is effective when it is actually not.³³ In exchange for this increased risk, fewer patients are required to reach the accrual goal, which makes it a popular approach.

Relaxing power.

A third approach to reducing sample size (albeit less common at COG) is to relax or decrease the amount of power or probability with which the targeted effect size will be detected if it is true (a power less than 80% usually is not considered). Although each of these adjustments comes with some corresponding statistical sacrifice, a compromise among the three objectives of low type I error, high power, and a reasonable targeted effect is often possible and straightforward to implement; thus, these approaches remain popular in rare disease settings.^23,27-39

Guidance for relaxing operating characteristics and trial parameters.

When trials include most of the patients in the target population, Sposto and Stram²³ investigated the relationship between type I error, sample size, trial duration, patient accrual rate, and therapeutic innovation rate and the increase in treatment efficacy achieved after a series of two-treatment randomized phase III trials. They proposed that “a more appropriate view of trial design in low-incidence cancer settings is in the overall context of the research setting and long-term goals rather than in the narrow context of the current single trial.”^(p1183) Sposto and Stram concluded that judicious choice of type I error and accrual duration will result in a larger average gain in treatment efficacy as measured by improvements in cure rates over the long term, especially in low-incidence disease, than would use of small type I error and high power for small differences. Their results also suggested that insistence on low error rates and performance of large trials in low-incidence cancer can be counterproductive for improving cure rates in the long term. Deley et al³⁹ and Bayar et al²⁸ did similar simulation studies and confirmed these findings. Strauss and Simon²⁹ considered the optimum allocation of patient resources to maximize success rates in a phase II context and had similar conclusions (ie, more frequent, smaller trials are warranted in settings where patient resources are limited). These articles support the notion that performance of a series of small trials with relaxed operating characteristics over a long-term research horizon leads (on average) to larger survival benefits. An optimal choice of design parameters can be selected to maximize the expected total survival benefit by considering factors such as accrual rate, therapeutic innovation rate, and disease severity.^23,28,39

Despite the justifications, the targeting of a larger effect size or relaxing of type I error has some disadvantages. Targeting a larger effect size with the type II error rate of 20% could lead to underpowered studies for new treatments associated with smaller, but still clinically meaningful benefits and, hence, increases the risk of missing a moderate or small treatment effect.^31,33 Use of larger type I error rates and/or shorter trials is associated with worse apparent treatment efficacy in the long term and an increased likelihood of ultimately selecting an inferior treatment compared with standard larger trials with lower type I error,²³ and increasing type I error (one-sided) above 20% generally is not recommended.^23,28,39

Examples of Relaxed Design Constraints and Solutions From COG Trials

At COG, many phase II and some phase III randomized studies were designed with relaxed type I error rates (one-sided 5%, 10%, or 15%) or were powered to target a larger effect size with a reduced statistical power of 80%. Table 2 lists some of these trials and includes their specific objectives and operating characteristics as examples. Among these is the study design for low-risk patients with relapsed B-ALL enrolled in the COG trial AALL1331 (ClinicalTrials.gov identifier: NCT02101853). The cohort-specific objective was to compare disease-free survival (DFS) between patients randomly assigned to chemotherapy alone (control arm) or chemotherapy plus blinatumomab (a bispecific single-chain antibody that targets the CD19 antigen) after re-induction. The corresponding study was designed to have 80% power to detect an HR of 0.55 using a log-rank test with 5% one-sided type I error, which corresponds to an increase in the 3-year DFS rate from 73% in the control arm to 84% in the experimental arm. The B-ALL cohort was expected to accrue during 5.6 years to reach 206 eligible patients, with the analysis requiring 3 years of additional follow-up after completion of enrollment. Like many COG trials, this trial used a relaxed type I error rate and targeted a larger treatment effect size to maintain feasibility. By doing so, we are able to research a larger number of treatment strategies in low-incidence cancer settings and continue our quest toward increased cure rates for our patients.

TABLE 2.

Example Children’s Oncology Group Clinical Trials or Clinical Trial Cohorts (Subtrials) Designed With Relaxed Type I Error Rates or Larger Targeted Effect Sizes (All With 80% Power)

EVALUATING NI

Motivation to Pursue NI Objectives in Pediatric Trials

Comparison of a novel treatment to placebo is typically not an option in pediatric cancer study settings; thus, new treatments in randomized trials typically must be compared with active control therapies.⁴⁰ As a result, effect sizes are expected to be smaller, which corresponds to a larger sample size, a challenge for rare diseases. Common objectives in randomized pediatric oncology trials include the comparison of drug combinations, altered doses, or adjuvant therapies against standard therapeutic backbones, with trials powered to detect improved outcomes in the experimental arms versus the control arm. Also common in pediatric trials are situations where the goal is not to prove superiority but to show that a new experimental therapy is not statistically inferior to an active control or standard of care and retains most (if not all) of the control’s therapeutic benefit. In this case, factors such as potential improvements in safety, tolerability, cost, convenience, availability, or other secondary factors are used to justify the study of the experimental regimen. This type of randomized study design is known as an NI trial. Despite their relevance in situations such as those just described, NI trials are often more challenging to design than superiority trials, and a number of factors must be considered during the planning stages to ensure a feasible trial capable of answering the NI-type question of interest.^41,42

Planning and Design Considerations for NI Trials

Overview of randomized NI designs using time-to-event end points.

The primary end point for randomized oncology trials is typically time to event, such as EFS measured since random assignment. As such, the results presented usually include a significance test for differences between groups (eg, a log-rank test) and a numerical summary of that difference (eg, the HR estimated from a Cox proportional hazards regression model). With the HR quantifying the relative risk of experiencing the outcome of interest (eg, relapse, death) in the experimental versus control arm such that an HR of 1 indicates equivalence, a typical randomized superiority design is powered to detect a reduction in relative risk (eg, HR of 0.70). In the randomized NI trial setting with a time-to-event outcome, however, interest lies in determining whether the estimated HR and its CI fall in such a region that inferiority of the experimental arm can be statistically ruled out. At the design stage, this region is defined when the NI margin (a fixed quantity on the HR scale greater than 1) is specified, thereby indicating the amount of increased relative risk that should not be exceeded by the upper limit of a one-sided CI for the HR.^41,42 For example, during trial planning, the NI margin for a particular comparison might be set at δ_M = 1.15, which suggests that the investigators would deem the experimental treatment noninferior to the control treatment so long as the upper bound of the CI for the HR lies below 1.15. At the time of the final analysis, if this condition is met, NI (or a stronger conclusion such as superiority) can be claimed for the experimental treatment (Fig 2).

FIG 2.

Some possible outcomes of a noninferiority design with a time-to-event end point, each displayed as a function of the CI for the HR (interval bar with dot estimate) and the prespecified noninferiority margin δ_M, with an HR of 1 indicating equivalence of the treatment arms.

General cautions about NI designs.

Although NI designs can be definitive if well executed, some words of caution are warranted. First, conclusions about the NI of a new regimen versus an active control are not particularly useful if the control was never established as best care in prior research.^43,44 In addition, if outcomes in the control group are better than anticipated during trial planning, a resulting NI study could be underpowered (even when the treatment arms are equivalent) because of an overall shortage of events among the enrolled patients. A typical NI design is powered under the assumption of risk equality between groups, although this can be adjusted in some cases (see herein the Challenges With Conducting NI Designs in Rare Disease Settings and Some Ad Hoc and Literature-Based Solutions section). In many cases, futility monitoring can be incorporated to halt the NI trial if it becomes evident that the experimental therapy is truly inferior, although specific testing approaches and decision rules should be considered carefully.^45-47

Challenges With Conducting NI Designs in Rare Disease Settings and Some Ad Hoc and Literature-Based Solutions

As is often the case in rare disease settings, the design of randomized NI trials for COG studies requires extra consideration and some degree of compromise to increase trial feasibility. This is especially true because NI trials tend to require larger sample sizes (on average) than trials for superiority, with all other features being approximately equal. A few such considerations and ad hoc solutions are described here.

Ad hoc selection of the NI margin.

The value of the NI margin δ_M generally is chosen on the basis of clinical consensus, the anticipated benefits of the experimental treatment versus the control, and the documented outcomes of patients currently receiving the standard of care.^42-44 However, little consensus exists with regard to NI margins that are reasonable, with reviews of recent oncology trials noting values of δ_M ranging from 1.1 to 1.5.^48,49 Furthermore, a 2015 Lancet Oncology commentary cautioned against arbitrary selection of the NI margin and other aspects of the design.⁵⁰ Although some combination of clinical judgment and arbitrary NI margin selection to preserve feasibility is a common strategy, a more-formal NI margin calculation methodology known as the effect retention methods has been proposed, although these methods depend on access to large sets of historical data for calibration.^44,51 In disease settings such as COG where smaller historical sample sizes and changes to complex treatment regimens across time are the norm, calculated measures derived from prior trials often are compromised or unavailable. In this case, straightforward specification of a somewhat higher NI margin is often necessary, subject to the buy-in of trial investigators and stakeholders. Within COG, all aspects of statistical design and supporting clinical rationale ultimately undergo several layers of review, during which the benefits and caveats are evaluated carefully with respect to the realistic constraints imposed by treatment options, the patient population, and other resources.

Relaxation of type I error.

Ideally, NI trials, especially those for regulatory indications, would specify no greater than a one-sided type I error rate of 2.5% by default to retain a high degree of rigor and safeguard against false-positive conclusions.^44,52 As in the superiority setting, a smaller type I error requires a larger sample size and widens the CI for the HR, making an NI conclusion much less likely to be a false-positive result. Within COG, NI trials usually are motivated by the desire to reduce treatment intensity in groups of patients with established good prognoses, thereby reducing short- or long-term toxicities among children expected to be cured. In this context, there is precedent for conducting NI trials with a one-sided type I error rate of 5% (or even more relaxed; see Relaxing Trial Operating Characteristics and Design Parameters section), which strikes a compromise between error rate conservation and simple feasibility.

Literature-proposed solutions for NI hypotheses in rare diseases.

Freidlin et al⁵³ adapted the NI design to allow for sample size calculations in situations where the assumed HR for the power calculation is not equal to 1 (equivalence) but reflects modest superiority for the experimental treatment (ie, HR slightly less than 1). This solution supports both a smaller required sample size and a reasonable choice of the NI margin (ie, not too large). In this scenario, an NI study might be underpowered to detect equivalence between the arms (HR of 1), but the ability to conclude NI is enhanced when a small improvement in outcome truly exists for the experimental arm. This is often a welcome compromise when the experimental arm already offers less toxicity or other advantages. In rare disease settings, such as pediatric cancer trials conducted by COG, this design often is considered when small treatment improvements are deemed plausible or likely (see Example Trials From COG Testing NI-Type Hypotheses section for examples). In the same article, Freidlin et al also described a sequential testing procedure that can be applied when a test for NI followed by a test for superiority (a higher bar) is desired.

Example Trials From COG Testing NI-Type Hypotheses

One recently opened COG trial with an NI design is AALL1631 (ClinicalTrials.gov Identifier: NCT03007147), an international collaborative study of the NI of an experimental, less-toxic postinduction treatment in patients with standard-risk Philadelphia chromosome–positive ALL.⁵⁴ Using historical data during trial planning, the 3-year DFS rate was predicted to be 70% in the control arm, and δ_M = 1.43 was set to correspond to a minimally reduced 3-year DFS rate of 60% in the experimental group. Assuming 6 years of accrual and a minimum of 3 years of follow-up, random assignment of 475 patients will yield 80% power to conclude NI using a one-sided type I error rate of 5%. Because of its long duration, this trial incorporates futility monitoring at three interim time points to detect true inferiority and prospective monitoring plans for treatment adherence and other study trends that could otherwise compromise study power.

Two other soon-to-open phase III COG studies also will use NI designs. ACNS1831 (ClinicalTrials.gov identifier: NCT03871257) and ACNS1833 (pending final protocol approval) will enroll patients with front-line NF-1–mutated and non–NF-1 low-grade gliomas, respectively, where the efficacy of selumetinib, an MEK inhibitor, will be compared against the current standard of care carboplatin plus vincristine. For patients with low-grade gliomas, the question of treatment de-intensification to avoid harmful adverse effects without sacrificing efficacy is highly relevant. However, given the rarity of this patient population, in addition to proposing a larger-than-usual NI margin (HR of 1.7) and somewhat inflated type I error rate of 10%, these two studies have used the design proposed by Freidlin et al⁵³ and described here in the Challenges With Conducting NI Designs in Rare Disease Settings and Some Ad Hoc and Literature-Based Solutions section to make these rare cancer NI trials more feasible because superiority is possible. The encouraging outcome data observed in the Pediatric Brain Tumor Consortium (PBTC)-029B study (ClinicalTrials.gov identifier: NCT01089101), which was conducted in the recurrent setting for the same patient population, provided the rationale for this design choice.^54,55

USE OF HISTORICAL CONTROLS

Motivation and Challenges for Use of Historical Controls in Pediatric Trials

Because of the rarity of the pediatric cancers and the fact that some outcomes have remained relatively unchanged in recent studies, several recent COG phase II trials have been designed to allow outcomes under the experimental treatment to be compared against outcomes previously observed from carefully constructed historical controls. Although historical cohorts have well-known weaknesses, including subject selection bias, evolving disease or diagnostic definitions, improvements in standard of care over time, or changes in disease evaluation criteria,^56-61 these and other concerns can be minimized by restricting consideration to recent studies conducted by the same cooperative group or where the characteristics of the patient population are otherwise well known. In addition, availability of individual patient data from the potential historical trials or data sources is key so that the formal cohort that resembles the control arm can be tailored to fit the current trial’s eligibility criteria and other characteristics. In cases where large historical control cohorts with adequate patient-level data are available, these designs can serve as an effective alternative to randomized trials in the context of signal finding. Our sense is that such approaches work best in circumstances where the outcome is dismal, the targeted efficacy signal is large, and the historical cohort available is deemed to be sufficiently similar to a cohort that would be enrolled if a concurrent control arm were included in the trial design.

Review of Trial Designs Using Historical Controls

Dixon and Simon⁶² described an initial approach to designing phase II studies using historical controls; however, it was subsequently shown by Korn and Freidlin⁶³ that this method does not adequately protect against type I or II errors. Instead, Korn and Freidlin proposed a two-sample randomized design approach with adjusted allocation to the size of the historical cohort, which optimizes estimation of the median sample size and controls the type I and II errors effectively. A drawback, however, is that the design is conservative in the sense that the number of events in the control cohort is almost always larger because of follow-up in the historical control being longer than what would have been observed with concurrent random assignment to cohorts of equal sample size. Another drawback of this approach is that calculation of accumulating end point information during the trial for the purposes of timing interim analyses is not readily possible as with traditional designs. Both drawbacks are addressed in an article by Wu and Xiong⁶⁴ that focused on the median sample size as the primary parameter to be optimized on the basis of the one-sided sequential log-rank test. Their method provides a closed-form formula for the number of events needed in the experimental arm and uses a transformed information time to accommodate appropriately the information from the historical control and experimental group together to enable interim analyses. Work by Wu and Li (manuscript submitted for publication) further extends this method for use with interim analyses on the basis of the alpha and beta spending functions of Lan and DeMets.

Example Trials From COG Using Historical Controls

A phase II study currently in development for newly diagnosed diffuse intrinsic pontine glioma will use pooled historical data from several similar completed studies as the comparison cohort, where the primary end point of interest is OS. As shown in Figure 3A, the OS outcomes from the last four published COG/CCG/POG trials are remarkably similar. This data set has a combined sample size of 212 participants, with only 10 who were right censored during follow-up. Using the methodology described in Wu and Xiong⁶⁴ and the empirical Kaplan-Meier estimate to model the combined OS distribution (Fig 3B), a reasonable phase II design will require 45 patients to target an HR of 0.60 with 80% power and maintain less than a 5% type I error rate using a one-sided log-rank test. This HR translates to an approximately 18% net improvement in 1-year OS rate (42% to 60%). Because the study accrual duration is expected to be 1 to 1.5 years, the design will require 1 year of additional follow-up for all patients after accrual completion. Interim monitoring for futility also is incorporated into the design. These design parameters reflect a compromise between what is feasible and what is clinically meaningful in this disease setting. A similar approach also has been used for ACNS1721 (ClinicalTrials.gov identifier: NCT03581292), which enrolls newly diagnosed patients with high-grade glioma at least 3 years of age whose tumors are risk stratified by molecular criteria at initial study screening. The primary end point of this trial is progression-free survival, and the efficacy comparison is based on a historical control cohort constructed from prior COG studies, including ACNS0423 (ClinicalTrials.gov identifier: NCT00100802) and ACNS0822 (ClinicalTrials.gov identifier: NCT01236560), as well as global meta-analysis cohorts where tumors received similar molecular annotation.⁶⁵

FIG 3.

Overall survival (OS) outcomes for historical Children’s Oncology Group studies in newly diagnosed diffuse intrinsic pontine glioma displayed (A) by study and (B) combined.

Recommendations for Use of Historical Controls

Although phase II designs that are based on historical control cohorts have desirable properties that support feasibility in rare disease settings, they are most useful when the available historical control group is relatively large. When the sample size and/or the number of events in the historical control cohort are small, the designs outlined here may not be feasible because the sample size in the proposed trial may not be able to compensate for the lack of patients/events supporting the historical cohort. On the other hand, if the historical control cohort is very large, then the design essentially reduces to a literature-controlled design where the comparison is against a fixed rate at a specified time point. Zhu et al⁶⁶ wrote an excellent discussion on this, including an alternative sample size calculation approach that is based on optimizing a prespecified percentile of the sample size. More sophisticated approaches for borrowing information from historical controls while simultaneously randomly assigning a small subset of patients to a control arm also have been described.⁶⁷

DISCUSSION

In this article, we highlighted some of the most prominent features and challenges that affect clinical trials designed for the pediatric oncology setting. Of note, most of the issues and potential solutions discussed herein are more broadly applicable to any rare cancer, including those that primarily affect adult patient populations. Because the statistical challenges we encounter continue to increase in both frequency and complexity in response to the dual international missions of personalized medicine and large-scale data sharing, it remains critical that statisticians and other researchers in rare disease settings continue to develop and promote novel approaches to trial design and analysis that are capable of adapting to the changing science of cancer while maintaining overall trial feasibility.

AUTHOR CONTRIBUTIONS

Conception and design: Lindsay A. Renfro, Lingyun Ji, Jin Piao, Arzu Onar-Thomas, Todd A. Alonzo

Collection and assembly of data: Arzu Onar-Thomas

Data analysis and interpretation: Lindsay A. Renfro, Arzu Onar-Thomas, John A. Kairalla, Todd A. Alonzo

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/po/author-center.

Arzu Onar-Thomas

Honoraria: Eli Lilly

Research Funding: Novartis (Inst), Apexigen (Inst), Pfizer (Inst), Celgene (Inst), Merck (Inst)

Travel, Accommodations, Expenses: Eli Lilly

John A. Kairalla

Stock and Other Ownership Interests: Johnson & Johnson, Sophiris Bio

No other potential conflicts of interest were reported.