Abstract
Purpose
Recent reports have suggested that 40% or more of National Cancer Institute (NCI) –sponsored Cooperative Group phase III trials failed to achieve their accrual goals. We examine in detail the accrual experience of the Cooperative Group phase III trials.
Patients and Methods
All Cooperative Group phase III trials activated from 2000 to 2007 were examined for their accrual experience. For trials that stopped accrual with < 90% of their accrual goal, the reasons for having < 90% accrual were documented. We focus on trials that ended with < 90% accrual because of inadequate accrual rates rather than for other reasons, such as an interim monitoring analysis by an independent data monitoring committee that stops the trial early because one treatment is clearly superior.
Results
There were 191 trials activated from 2000 to 2007. We project that 22.0% of these trials will have < 90% accrual because of inadequate accrual rates. We project that there will be 176,627 patients eventually accrued on the 191 trials (current accrual, 154,579) and that 2,991of these patients will be on trials that have < 90% accrual because of inadequate accrual rates (1.7%). For nonpediatric cancer trials, the corresponding percentages are 26.7% and 2.0%.
Conclusion
We find that insufficient accrual rates are not as high as previously reported and that only a small proportion of patients were enrolled on trials that ended with insufficient accrual because of an inadequate accrual rate. NCI has implemented new procedures to reduce the number of trials that fail to reach their accrual goals and to minimize the number of patients accrued on these trials.
INTRODUCTION
The Cancer Therapy Evaluation Program (CTEP) of the National Cancer Institute (NCI) supports phase III clinical trials primarily through the NCI Clinical Trials Cooperative Group program. There are currently approximately 100 such trials actively accruing patients. Although many Cooperative Group phase III trials have led to major advances in the treatment of cancer patients,1 a proportion of trials are never completed because they do not achieve a sufficient accrual to meet their scientific objectives. Such trials represent loss of the resources that went into designing the trials, getting them activated, and treating the patients accrued on the trials, as well as not utilizing the efforts of the participating patients. In addition, an ongoing trial may preclude opening other trials in the same disease setting that might have been successfully completed. As part of ongoing efforts to improve the efficiency of the NCI clinical trials program, we have performed an in-depth review of the accrual experience from Cooperative Group phase III trials activated from 2000 to 2007.
PATIENTS AND METHODS
Trial Information
All CTEP-supported phase III trials led by an NCI-sponsored Cooperative Group or conducted as part of an international collaboration with a Cooperative Group that were activated in the years 2000 to 2007 were identified. Trials were categorized as having accrual finished or not, with the latter category including trials that were temporarily closed to accrual. The accrual goal of the trial was taken from the latest CTEP-approved version of the trial protocol. The percent accrued for the trial was calculated as the current or final accrual divided by the accrual goal of the trial. Closed trials with < 90% accrued were considered not fully accrued. Trials not fully accrued were categorized for the reason they stopped accruing by using the following categories: (a) external information (eg, results of another clinical trial that answered the current trial question or rendered it irrelevant), (b) formal interim monitoring of the current trial by an independent data monitoring committee (either for showing one of the trial arms superior or for futility/inefficacy of the experimental treatment arm), (c) unacceptable toxicity, (d) drug supply issues, or (e) inadequate accrual rate. Information for performing the categorization was obtained from administrative documents (eg, protocol amendments and protocol status updates), trial publications, and CTEP investigators. Trials were additionally categorized by the primary disease site, whether or not the trial involved a randomization (some pediatric phase III trials use historical controls), and whether or not the trial involved an investigational new drug agent.
Statistical Analyses
Trials that were closed to accrual with < 90% of their accrual goal were considered to have insufficient accrual, with the 90% figure chosen prospectively before the analysis was begun. Considering that the statistical power of a trial is typically based on an estimated number of events that will be observed (which depends on the length of the follow-up), we believe that trials that achieve ≥ 90% of their accrual goal can be considered successfully accrued from a statistical point of view. One parameter of interest is the probability that a trial activated from 2000 to 2007 will have insufficient accrual because of an inadequate accrual rate (category (e) above). Since not all trials activated from 2000 to 2007 have completed accrual, this parameter needs to be estimated. If one estimates solely from trials that have closed to accrual, then the estimator will be subject to sampling bias. (This is the same type of bias one would observe by trying to estimate median patient survival in a clinical trial by using the median survival of only those patients who have died.) To avoid sampling bias, statistical methods for survival data that account for censored observations were used that adjust for actively accruing trials. In particular, (1) the unit of analysis is the trial, (2) “time” on study is the percentage accrued for the trial, (3) the trial is considered as having the “event of interest” if the trial stopped accruing with < 90% accrued because of inadequate accrual, (4) trials that stopped with < 90% accrual for other reasons (eg, interim monitoring) are considered a competing risk, and (5) trials that are still actively accruing are treated as censored observations. (Trials that have accrual temporarily suspended are considered active). In this framework, the parameter of interest is the crude cumulative incidence2 evaluated at 90% percent accrued.
Other parameters of interest are a projection of the number of patients who will be accrued to trials that will have insufficient accrual because of an inadequate accrual rate and the proportion of such patients compared with the total number of patients who will be accrued to all trials. To obtain estimators of these parameters, survival methods were applied with the analyses weighted by the accrual goal for each trial (details are found in the Appendix, online only).
RESULTS
One hundred ninety-one phase III trials were activated from 2000 to 2007 (Table 1). Figure 1 displays a histogram of the percentage accrued for the 133 trials for which accrual has finished; trials having an inadequate accrual rate are shown in gold. The estimate of the proportion of trials that have insufficient accrual because of an inadequate accrual rate is 22.0%. This estimate is remarkably similar to the naive proportion of trials that had an inadequate accrual rate (21.5%; 41 of 191), which would be an appropriate estimator if we knew that all actively accruing trials would eventually achieve at least 90% accrual. The reason for this is that practically all of the actively accruing trials are already past the point (in terms of percent accrued) at which trials that are going to stop because of inadequate accrual would have stopped (Fig 2). In particular, 85% (35 of 41) of the trials that closed for inadequate accrual rates had < 20% accrued (Fig 1), and 91% (53 of 58) of the trials still accruing already have > 20% accrued (Fig 2).
Table 1.
Status | No. of Trials |
---|---|
Accrual not over | 58 |
≥ 90% accrued so far | 9 |
< 90% accrued so far | 49 |
Accrual over | 133 |
≥ 90% accrued | 68 |
< 90% accrued | 65 |
Reasons for < 90% accrued | |
Interim monitoring | 12*† |
External information | 9* |
Drug supply issues | 2 |
Unacceptable toxicity | 3 |
Inadequate accrual rate | 41 |
Abbreviation: CTEP, Cancer Therapy Evaluation Program.
Includes two trials that had < 90% accrued because of both interim monitoring and external information.
Two of the 12 trials were stopped for superiority monitoring; the other 10 were stopped for futility monitoring.
The estimate of the proportion of patients enrolled on trials that had insufficient accrual because of an inadequate accrual rate is 1.7%, representing a projected 2,991 patients of a projected 176,627 that will eventually be accrued to all 191 trials. This low percentage reflects the obvious point that trials stopped for inadequate accrual will tend to have only a small number of patients accrued.
When examined by primary disease site (Table 2), the pediatric cancer trials have a smaller proportion of trials with inadequate accrual rate leading to < 90% accrued than the adult cancer trials. In fact, only two of the 42 pediatric trials had < 90% accrued because of an inadequate accrual rate. For the adult cancer trials, the breast cancer trials appear to have fewer trials with inadequate accrual rates. None of the 15 phase III trials with nonrandomized designs had < 90% accrued because of an inadequate accrual rate (Table 3); these trials were all pediatric cancer trials. There is no substantial difference in the proportion of inadequately accruing trials depending on whether or not the trial involved an investigational new drug agent (Table 3).
Table 2.
Primary Disease Site | Trials |
Patients |
||||
---|---|---|---|---|---|---|
No. Activated | Estimated Proportion With Inadequate Accrual Rate (%)* | No. of Patients Accrued to Date | Projected No. Accrued When All Trials Are Closed*† | Projected No. on Trials With Inadequate Accrual Rate*† | Estimated Proportion on Trials With Inadequate Accrual Rate (%)* | |
Adult | ||||||
Breast | 31 | 13.0 | 69,936 | 76,382 | 362 | 0.5 |
Hematopoietic | 25 | 28.8 | 6,795 | 7,108 | 240 | 3.4 |
GI | 18 | 28.6 | 18,437 | 20,316 | 746 | 3.7 |
Female reproductive | 16 | 37.5 | 10,174 | 11,304 | 193 | 1.7 |
Lung | 14 | 22.6 | 5,652 | 7,198 | 640 | 8.9 |
Prostate | 16 | 25.0 | 8,951 | 11,204 | 150 | 1.3 |
Other‡ | 29 | 34.9 | 10,988 | 11,470 | 523 | 4.6 |
Subtotal | 149 | 26.7 | 130,933 | 147,742 | 2,930 | 2.0 |
Pediatric | ||||||
Nonhematopoietic | 26 | 7.7 | 6,024 | 8,845 | 25 | 0.3 |
Hematopoietic | 16 | 0.0 | 17,622 | 19,471 | 0 | 0.0 |
Subtotal | 42 | 4.8 | 23,646 | 28,955 | 28 | 0.1 |
Total | 191 | 22.0 | 154,579 | 176,627 | 2,991 | 1.7 |
These proportions and numbers are estimated using survival analysis methodology.
Because projected numbers for subgroups are based on within-subgroup survival curves, numbers do not add up exactly to totals.
Includes seven head and neck cancer, four distant metastases (unspecified origin), three melanoma, three astrocytoma, three renal, three soft tissue sarcoma, three bladder, one testicular, one neuroendocrine, and one breast/colorectal cancer trials.
Table 3.
Randomization-in-Design Status | Trials |
Patients |
||||
---|---|---|---|---|---|---|
No. Activated | Estimated Proportion With Inadequate Accrual Rate (%)* | No. of Patients Accrued to Date | Projected No. of Patients Accrued When All Trials Closed*† | Projected No. on Trials With Inadequate Accrual Rate*† | Estimated Proportion on Trials With Inadequate Accrual Rate (%)* | |
Randomized design | ||||||
IND agent | 60 | 18.5 | 50,947 | 59,174 | 1,073 | 1.8 |
No IND agent | 116 | 26.7 | 100,882 | 112,716 | 1,864 | 1.7 |
Subtotal | 176 | 23.9 | 151,829 | 172,488 | 2,983 | 1.7 |
Nonrandomized design | 15 | 0.0 | 2,750 | 4,370 | 0 | 0.0 |
Total | 191 | 22.0 | 154,579 | 176,627 | 2,991 | 1.7 |
Abbreviation: IND, investigational new drug.
These proportions and numbers are estimated using survival analysis methodology.
Because projected numbers for subgroups are based on within-subgroup survival curves, numbers do not add up exactly to totals.
Although ≥ 90% accrual was the prospectively defined cutoff for sufficient accrual, similar results are obtained by using a 95% cutoff. Using a 95% cutoff, the estimate of the proportion of trials that have insufficient accrual because of an inadequate accrual rate is 22.8% (instead of 22.0%), and the estimate of the proportion of patients enrolled on trials that had insufficient accrual because of inadequate accrual rate is 2.2% (instead of 1.7%).
DISCUSSION
Cheng et al3 report 49.2% (30 of 61) of CTEP-approved nonpediatric phase III trials failed to achieve at least 25% of accrual goals. Recently, the Institute of Medicine reported that 40% of CTEP-approved phase III trials failed to achieve minimum accrual goals,4 a figure that has been repeated elsewhere.5–7 We report here that we estimate that 28.3% of such nonpediatric trials will fail to achieve at least 90% of their accrual goals because of inadequate accrual, based on data from 149 trials (Table 2). The difference between the results can be attributed to exclusion of actively accruing trials by Cheng et al3 (leading to sampling bias) and their inclusion, as failures to achieve accrual goals, of trials that ended for other reasons besides inadequate accrual.4 We have chosen not to consider trials that failed to achieve at least 90% of their accrual goals because of formal interim monitoring, unacceptable toxicity, or drug supply issues as failures. This is an obvious decision for trials that closed because of interim monitoring, and one could argue for the other categories that failure to fully accrue was beyond the control of the investigators.
Overall, we estimate that 22.0% of all trials (adult and pediatric) will end with insufficient accrual because of inadequate accrual rates, and 1.7% of the total number of patients accrued on all trials will be on these trials. It is possible that a trial that ends with accrual < 90% of projected because of an inadequate accrual rate can still provide useful clinical information. For example, the Eastern Cooperative Oncology Group (ECOG) E4201 trial,8 which closed to accrual with 74 of 332 patients accrued, demonstrated the major advance of treating locally inoperable pancreatic cancer with radiation therapy in addition to gemcitabine.9 Another example is given by the Radiation Therapy Oncology Group (RTOG) 9813 trial, which closed to accrual with 201 of 454 patients accrued and is in follow-up. This trial, which compares temozolomide plus radiation versus nitrosourea plus radiation for treating anaplastic astrocytomas or mixed gliomas, may still provide relevant clinical information. Much more likely, trials that end early because of inadequate accrual will provide little useful clinical data. Although the number of patients involved in these trials is small (compared with the number of patients on all trials), there are still considerable resources involved in opening a trial, whether or not it accrues.
Should one aim for a clinical trials program to open only trials in which one is positive that they will accrue successfully? We would argue no, because this would preclude starting trials that address important questions but in which it is known at the start that accrual will be challenging. For example, the Surgical Prostatectomy Versus Interstitial Radiation Intervention Trial (SPIRIT; American College of Surgeons Oncology Group [ACOSOG] Z0070), comparing radical prostatectomy versus brachytherapy in early-stage prostate cancer, accrued only 56 of a required 1,980 patients. Another example is given by the Southwest Oncology Group S0521 trial, which compared maintenance chemotherapy versus observation in patients with previously untreated low- and intermediate-risk acute promyelocytic leukemia. It accrued only 95 of a required 500 patients. Yet experts often cite the strong need for clinical trials for both these questions.
Although we believe it is important to attempt to perform important trials that may be a challenge for enrollment, it is also important to minimize the time and number of patients involved in trials that turn out to have insufficient accrual. One strategy is to examine characteristics of inadequately accrued trials to help inform trial prioritization.10
A second strategy is to open a trial first in a limited number of institutions to assess accrual feasibility. This strategy was used in the Surveillance Therapy Against Radical Treatment (START) trial (National Cancer Institute of Canada Clinical Trials Group [NCIC CTG] PR.11), testing radical prostatectomy or radiotherapy versus active surveillance for favorable-risk prostate cancer.
A third strategy is to stop trials early when it is apparent they will never reach their accrual goals because of inadequate accrual rates. To this end, we developed CTEP early-stopping guidelines11 that apply to slow-accruing phase III Cooperative Group trials activated after April 1, 2004 (trials that have < 20% of their projected accrual rates in quarters 5 and 6 after their activation are closed). (Twenty-six of the 41 trials that had inadequate accrual rates in Table 1 were activated before April 1, 2004.) These guidelines were based on historical data that demonstrated that trials with poor accrual in this time interval would be extremely unlikely to ever reach their accrual goals.12 Our experience with the CTEP early-stopping guidelines will be reported when we have further follow-up of the trials activated after April 1, 2004.
A fourth strategy is to simplify the enrollment process and expand patient entry onto trials. To this end, CTEP has developed the Cancer Trials Support Unit (CTSU),13 which allows for a larger number of institutions to enter patients into different Cooperative Group trials in an efficient manner. This strategy appears to be successful, because CTEP data (not shown) indicate that cross-Group accrual (enrollments from Groups other than the lead Group) has increased from an average of 20% in the pre-CTSU 1990s to 40% in the post-CTSU 2000s.
A fifth strategy is to simplify the data collection required for patients on trials, which may encourage physicians to participate. CTEP is working with the US Food and Drug Administration to reduce certain types of adverse event reporting, which may help in this regard.14
Finally, because slow development of a trial concept to a protocol ready for enrollment is associated with its ability to achieve its accrual goal,3 CTEP, working in concert with the Cooperative Groups, developed the Central Institutional Review Board for faster protocol review15 and has recently instituted new timelines for all phases of trial development.16 The target timelines to move from a trial concept to a protocol ready for accrual for phase II and III Cooperative Group trials have been reduced to 7 and 10 months, respectively, a > 50% reduction from current timelines. If these target timelines are achieved, then we will be able to determine whether this promising approach is indeed successful in reducing the number of trials that fail to meet their accrual goal.
Acknowledgment
We thank James R. Anderson, Karla V. Ballman, Mark F. Brady, Joseph P. Constantino, James Dignam, Stephen L. George, Robert Gray, Daniel J. Sargent, and Dongsheng Tu for their helpful comments.
Appendix
Detailed Statistical Methods for Estimating Numbers and Proportions of Patients
For the ith trial, let Ni be the accrual goal (from the protocol), let Ci be the current percent accrued of the accrual goal, let Yi0 be the percent accrued of the accrual goal when the trial stops accruing (which is unknown for trials that are actively accruing), and let Ri0 be the reason the trial stopped accruing for trials that stopped accruing with < 90% of their accrual goal (which is also unknown for trials that are actively accruing). For K = 191 trials, we estimate:
(1) total projected accrual = Σi = 1K Ni Yi0
(2) total projected accrual on trials with < 90% accrued that stopped accruing because of inadequate accrual rates = Σi = 1K Ni Yi0 I(Yi0 < 90% and Ri0 = “inadequate accrual rate”)
(3) proportion of patients on trials in (2) = ratio of (2) divided by (1)
where I(.) is the indicator function that is 1 if the argument is true and 0 otherwise. We estimate (1) and (2) by Σi = 1K Ni Yi and Σi = 1K Ni Yi Pi, respectively, where, for trials that have stopped accruing, Yi = Ci and Pi = 1 if the trial stopped accrual with < 90% accrued because of an inadequate accrual rate and 0 otherwise. For trials that have not stopped accruing, we use survival analysis methodology to estimate the Yi and Pi as follows: To estimate Yi, we construct a weighted Kaplan-Meier plot by using percent accrued as the time scale, with events being permanent closure of accrual for any reason, with trials still accruing being censored with their current percent accrued, and the weights being the accrual goals (Ni). From this Kaplan-Meier plot we estimate Yi = E(Yi0|Yi0>Ci). To estimate Pi, we construct a weighted cumulative incidence plot by using percent accrued as the time scale, with the plot conditional on Yi0>Ci, the event of interest being closure of accrual with < 90% accrued for an inadequate accrual rate, and the weights being the accrual goals (Ni). From this cumulative incidence plot, we estimate Pi to be the cumulative incidence at 0.9. Because this is a nonstandard use of survival analysis methodology, the results should be viewed as approximate. Interestingly, the proportion (3) derived by using this method is almost the same as the ratio of the current number of patients on trials that ended with < 90% accrued because of insufficient accrual divided by the current number accrued to date. For example, overall there are 154,579 patients accrued to date on the 191 trials, of which 2,666 were on 41 trials with < 90% accrued because of insufficient accrual. The ratio 1.7% (2,666 of 154,579) is the same (within rounding) to the 1.7% calculated figure (last row of Table 2).
Footnotes
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
The author(s) indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
Provision of study materials or patients: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
Collection and assembly of data: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
Data analysis and interpretation: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
Manuscript writing: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
Final approval of manuscript: Edward L. Korn, Boris Freidlin, Margaret Mooney, Jeffrey S. Abrams
REFERENCES
- 1.Zuckerman B, Corrigan J. The role of NCI-sponsored research in ASCO Clinical Cancer Advances 2005-2008. J Clin Oncol. 2010;28(suppl):470s. abstr 6094. [Google Scholar]
- 2.Korn EL, Dorey FJ. Applications of crude incidence curves. Stat Med. 1992;11:813–829. doi: 10.1002/sim.4780110611. [DOI] [PubMed] [Google Scholar]
- 3.Cheng S, Dietrich M, Finnigan S, et al. A sense of urgency: Evaluating the link between clinical trial development time and the accrual performance of CTEP-sponsored studies. J Clin Oncol. 2009;27(suppl):325s. doi: 10.1158/1078-0432.CCR-10-0133. abstr CRA6509. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Institute of Medicine. A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. Washington, DC: The National Academies Press; 2010. p. 143. [PubMed] [Google Scholar]
- 5.Editorial: Faltering cancer trials. New York Times, April 24, 2010 (a version of this editorial appeared on p WK11 of the New York edition on April, 25, 2010) http://www.nytimes.com/2010/04/25/opinion/25sun1.html.
- 6.Young RC. Cancer clinical trials: A chronic but curable crisis. N Engl J Med. 2010;363:306–309. doi: 10.1056/NEJMp1005843. [DOI] [PubMed] [Google Scholar]
- 7.Stewart DJ, Whitney SN, Kurzrock R. Equipoise lost: Ethics, costs, and the regulation of cancer research. J Clin Oncol. 2010;28:2925–2935. doi: 10.1200/JCO.2009.27.5404. [DOI] [PubMed] [Google Scholar]
- 8.Loehrer PJ, Powell ME, Cardenes HR, et al. A randomized phase III study of gemcitabine in combination with radiation therapy versus gemcitabine alone in patients with localized, unresectable pancreatic cancer: E4201. J Clin Oncol. 2008;26(suppl):214s. doi: 10.1097/COC.0b013e3181e9c103. abstr 4506. [DOI] [PubMed] [Google Scholar]
- 9.Winer E, Gralow J, Diller L, et al. Clinical cancer advances 2008: Major research advances in cancer treatment, prevention, and screening—A report from the American Society of Clinical Oncology. J Clin Oncol. 2009;27:812–826. doi: 10.1200/JCO.2008.21.2134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Schroen AT, Petroni GR, Wang H, et al. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clin Trials. 2010;7:312–321. doi: 10.1177/1740774510374973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Cancer Therapy Evaluation Program (CTEP): CTEP-Sponsored Cooperative Group Phase III trials: Early Stopping Guidelines for Slow Accruing Trials. http://ctep.cancer.gov/protocoldevelopment/docs/slow_accrual.pdf.
- 12.Cancer Therapy Evaluation Program (CTEP): Development of Early Stopping Guidelines for Slow Accruing Trials. http://ctep.cancer.gov/protocoldevelopment/docs/slow_accrual_guidelines_dev.pdf.
- 13.Cancer Trials Support Unit (CTSU) https://www.ctsu.org/public/
- 14.Engelberg Center for Health Care Reform at Brookings. 2009 Conference on Clinical Cancer Research Event Summary. http://www.brookings.edu/∼/media/Files/events/2009/0914_clinical_cancer_research/Event%20Summary.pdf.
- 15.Wagner TH, Murray C, Goldberg J, et al. Costs and benefits of the National Cancer Institute Central Institutional Review Board. J Clin Oncol. 2010;28:662–666. doi: 10.1200/JCO.2009.23.2470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Operational Efficiency Working Group. Report of the Operational Efficiency Working Group of the Clinical Trials and Translational Research Advisory Committee—Compressing the Timeline for Cancer Clinical Trial Activation, March 2010. http://ccct.cancer.gov/files/OEWG-Report.pdf. [Google Scholar]