Implementation of Timeline Reforms Speeds Initiation of National Cancer Institute–Sponsored Trials

Abstract

Background

The National Cancer Institute (NCI) organized the Operational Efficiency Working Group in 2008 to develop recommendations for improving the speed with which NCI-sponsored clinical trials move from the idea stage to a protocol open to patient enrollment.

Methods

Given the many stakeholders involved, the Operational Efficiency Working Group advised a multifaceted approach to mobilize the entire research community to improve their business processes. New staff positions to monitor progress, protocol-tracking Web sites, and strategically planned conference calls were implemented. NCI staff and clinical teams at Cooperative Groups and Cancer Centers strived to achieve new target timelines but, most important, agreed to abide by absolute deadlines. For phase I–II studies and phase III studies, the target timelines are 7 months and 10 months, whereas the absolute deadlines were set at 18 and 24 months, respectively. Trials not activated by the absolute deadline are automatically disapproved.

Results

The initial experience is encouraging and indicates a reduction in development times for phase I–II studies from the historical median of 541 days to a median of 442 days, an 18.3% decrease. The experience with phase III studies to date, although more limited (n = 25), demonstrates a 45.7% decrease in median days.

Conclusions

Based upon this progress, the NCI and the investigator community have agreed to reduce the absolute deadlines to 15 and 18 months for phase I–II and III trials, respectively. Emphasis on initiating trials rapidly is likely to help reduce the time it takes for clinical trial results to reach patients in need of new treatments.

The Cancer Therapy Evaluation Program (CTEP) of the National Cancer Institute (NCI) has long sponsored an oncology drug development effort whose size and scope is unique among National Institutes of Health institutes. NCI’s program complements commercial pharmaceutical plans by expanding research opportunities and by filling gaps in drug development that are unmet by the private sector. When new agents are ready for the clinic, NCI forms partnerships with either industry or academic institutions to bring their new molecules to NCI-sponsored clinical trial networks. NCI’s role in these studies may include filing an investigational new drug (IND) application with the US Food and Drug Administration, distributing experimental agents to trial sites, reviewing the experimental protocol, and monitoring adverse event reports on NCI IND agents to assure patient safety. Investigators in NCI-supported networks are responsible for proposing ideas, developing protocols, and conducting the trials, including data management, analysis, and publication. Both early phase I–II trials and later phase III randomized, controlled trials are conducted in NCI-supported Comprehensive Cancer Centers, other academic medical centers, community practices, and NCI-sponsored cooperative groups and consortia across the country.

Although fully acknowledging that NCI-sponsored trials have made many important contributions to oncology practice, several recent independent reviews (1,2) expressed concern over the inordinately long time required for implementing these studies because of the multilayered, sequential review processes. Overly long development can render trials outdated by the time they are available for patient enrollment and can considerably delay bringing new treatments to patients in need (1). If delays in activating trials were not addressed, the reviews (1–4) warned that the overall NCI clinical trials effort could be severely compromised (1–4).

These concerns led the NCI to reassess how its clinical trials were developed, reviewed, and implemented. Sponsors, government regulators, funders, patient advocates, and investigators all have responsibilities that traditionally have led to multiple, iterative review processes before a trial is launched (5). Recognizing that new approaches were urgently required, NCI formed an Operational Efficiency Working Group (OEWG) in 2008 that included all the critical stakeholders involved in the protocol development and approval process for NCI-supported trials. The March 23, 2010 OEWG report, “Compressing the Timeline for Cancer Clinical Trial Activation,” provided comprehensive strategies to decrease the time required to activate NCI-sponsored clinical studies (3). Although this effort took nearly 2 years to complete, it resulted in a broad-based, strategically driven plan that involves all the critical stakeholders in the cancer clinical trials community.

One essential OEWG recommendation was the establishment of both target timelines and absolute deadlines for trials as they transit from submission of an idea to trial activation (trial activation is defined as “open to patient enrollment at one or more institutions”). The target timeline was intended to represent an optimal scenario where each step in the process was achieved with great efficiency and resulted in very rapid protocol implementation (Figure 1). The absolute deadline is the maximum number of (calendar) days a trial is allotted from submission of a proposal to trial activation. For phase I–II studies and phase III studies, the target timelines are 7 months and 10 months, whereas the absolute deadlines were set at 18 and 24 months, respectively. Trials that do not activate before the absolute deadline are automatically disapproved. The target timeline, however, is the primary goal that NCI staff and the extramural investigators strive to achieve. It was agreed that target timelines for early- and late-phase studies needed to be aggressive and required a major improvement over past performance. OEWG members representing the pharmaceutical industry felt that these target timelines should be consistent with expectations in industry-sponsored studies.

Figure 1.

Key steps in the National Cancer Institute (NCI) clinical trial review process. FDA = US Food and Drug Administration; IRB = institutional review board; LOI = letter of intent.

Methods

Implementation Plan

Implementing the OEWG recommendations required that NCI and its trial networks overhaul the entire protocol review process, an effort that resulted in staffing additions, process improvements, and information technology innovations (Table 1).

Table 1.

Comprehensive changes undertaken to improve trial initiation treatments

Component	Change	Implementation
Target timeline	An ideal goal, achievable if all partners function optimally	7 months for phase I–II trials and 10 months for phase III trials
Absolute deadline*	An immoveable date by which the trial must be open to patient enrollment	18 months for phase I–II trials and 24 months for phase III trials*
Staffing additions	New positions created to manage protocol timelines and to assist physicians with protocol authorship, revisions, and editing
Process improvement	Implementation of uniform templates for protocol development and for reviewers’ comments	Requirement for prompt teleconferences to resolve scientific and regulatory review issues at each step of review
Information technology	Creation of a website to track all phases of protocol’s life cycle

* The absolute timelines were revised to be more stringent: 15 months for phase I–II trials and 18 months for phase III trials.

Staffing Additions.

At NCI, two new roles were established: project manager and medical editor. The project managers facilitate all aspects of the protocol lifecycle, from initial receipt of the proposal until trial activation. They identify roadblocks during trial development and work with NCI staff and extramural investigators to assess how to best remedy problems that may develop during the protocol review process. The role of the NCI medical editors is to increase the efficiency of the protocol revision process. They assist NCI staff and extramural investigators by compiling and editing protocol documents and reviews.

Cancer centers and cooperative groups also established new positions to improve protocol development timelines. At Mayo Clinic and the University of Texas MD Anderson Cancer Center, a quality improvement coordinator and NCI support services director, respectively, were hired to oversee timelines; the National Surgical Adjuvant Breast and Bowel Project (NSABP) added regulatory and protocol specialists and also made timeline compliance a prime responsibility of the group’s associate director. The Alliance (a new cooperative group consisting of the former Cancer and Leukemia Group B, the North Central Cancer Treatment Group, and the American College of Surgeons Oncology Group) created a protocol project manager position to specifically report on progress for each study against OEWG target timelines. The Eastern Cooperative Oncology Group (ECOG) has piloted the use of medical writers to assist study chairs in rapidly developing and assembling protocols, and the Radiation Therapy Oncology Group (RTOG) hired new project administrators who focus on bringing all relevant parties together rapidly to resolve problems.

Process Improvement.

The development of a protocol begins with the submission of trial ideas by the trial networks to NCI; these are termed “letters of intent” (LOIs) for early phase trials (phase I and phase II) or “concepts” for phase III trials (including phase II/III trials) conducted by the NCI-sponsored cooperative groups or consortia. These LOIs/concepts represent abbreviated versions of the eventual trial that allow reviewers to determine whether the investigator has a proposal that merits development into a full protocol document. The overall OEWG target timeline from LOI/concept submission to trial activation for early-phase trials is 210 days and for phase III trials is 300 days. Breaking the overall timeline into three distinct stages provides intermediate targets to monitor success throughout the process (Figure 1). The first stage has a target timeline of 60 days for LOIs and 90 days for concepts from submission of the LOI/concept to approval by the NCI; concepts are given an additional 30 days because they must be reviewed by NCI disease-specific steering committees that meet only monthly. This stage includes review of the proposal, generation of a consensus review document, a conference call between NCI and the study investigators to discuss the review, and approval by industry partners for NCI-held IND agents. After LOI/concept approval, the second stage has a target of 60 days for early-phase trials and 90 days for phase III trials, during which investigators develop and submit the full protocol document. The third and final stage targets the time from receipt by NCI of the full protocol document until trial activation; it should not exceed 90 days for early-phase trials and 120 days for phase III trials. During this time, the full protocol also undergoes NCI review with generation of a consensus review document and a conference call between NCI and investigators. In addition, regulatory and contract issues are addressed, the protocol undergoes institutional review board (IRB) review, and the trial is opened to patient enrollment.

To expedite the review process, NCI created consensus review templates to transmit reviewer comments to investigators. Whenever possible, medical editors convey comments with language suitable for insertion directly into the protocol document using the “track changes mode” in the word processing software, thereby eliminating redundant effort on the part of investigators.

In addition to these administrative changes, NCI and the trials networks agreed to promptly schedule conference calls between NCI staff and the investigator teams. The calls are scheduled after both LOI/concept and protocol review to promote rapid resolution of critical issues and serve to replace the previous, slower process of exchanging written comments. Since this process was instituted in April 2010, more than 550 conference calls have been held between NCI and extramural investigators.

To complement the system-wide changes made by NCI, cooperative groups and cancer centers have also changed their internal approaches. The Alliance and RTOG, for example, have eliminated steps in their internal review, mandated firm timelines for others, and scheduled conference calls to assure that their teams achieved consensus in advance of the NCI calls. Review of NSABP internal processes resulted in a decision to involve key staff responsible for protocol forms and computer programming much earlier, soon after concept approval, than was formerly the case. SWOG developed a specific intervention termed “RaPID” (Re-engineering Protocol Implementation and Development) whereby, for complex or urgent trials, a single 3-hour meeting of the study team is held in person. For simpler studies, the meeting is a Web-supported teleconference. This approach has been well received by the study teams and has not resulted in a substantial increase in costs. SWOG has decided to expand this RaPID process to all their studies as the initial pilot reduced activation time by a median of 1 year compared with trials activated during the same timeframe that did not use this new process. A different approach has been taken by ECOG, which decided to begin protocol document preparation, regulatory processing, and contract negotiations soon after a concept receives internal approval rather than await NCI approval. To compress timelines and avoid sequential reviews, the University of Texas MD Anderson Cancer Center now seeks parallel reviews by institutional scientific review committees and by NCI. In some cases, an NCI approval letter can result in exemption from internal review. The Moffitt Cancer Center has sped up regulatory review by sending protocols for scientific review and IRB review concurrently instead of sequentially. This has not led to endless iterative loops, despite initial concerns, because protocols approved by NCI have needed few changes.

Information Technology Innovations.

A new, password-protected, role-based website was developed by NCI in response to the OEWG recommendations that allows both NCI personnel and investigators and staff at trial networks to track the timelines of their trials. It is now possible to know at any time where a trial is in its development path and who has the responsibility to move it forward. The tracking website provides detailed information for each protocol development milestone and provides a running timeline that permits comparison with the target timeline.

Data Analysis

To compare pre- and post-OEWG development times of protocols, we restrict attention to approved LOI/concepts submitted in the period from 2004 to 2008 (pre-OEWG trials) and approved LOI/concepts submitted between April 1, 2010, and May 1, 2012 (post-OEWG trials), as shown in Table 2. To avoid underestimating development times for the post-OEWG trials, a small number of trials with LOIs/concepts still in review are considered as having their LOI/concept approved. For estimating the time-to-event distributions, events were considered censored observations if they had not occurred by April 1, 2010, for the pre-OEWG trials and by July 30, 2012, for the post-OEWG trials. Events were also considered censored if the protocol was withdrawn or disapproved, with the censoring date being the date of withdrawal or disapproval.

Table 2.

Early- and late-phase trials submitted to the National Cancer Institute

Types of trials	Pre-OEWG trials (2004–2008), No. (%)	Post-OEWG trials (April 1, 2010–May 1, 2012), No. (%)
Early-phase trials*
LOI submitted	1623	459
LOI approved	525 (32.3)	152 (33.1)
LOI disapproved	958 (59.0)	259 (56.4)
LOI withdrawn	140 (8.6)	41 (8.9)
LOI in review	0 (0)	7 (1.5)
Late-phase trials*
Concepts submitted	187	52
Concepts approved	112 (59.9)	24 (46.2)
Concepts disapproved	56 (29.9)	17 (32.7)
Concepts withdrawn	19 (10.2)	10 (19.2)
Concepts in review	0 (0)	1 (1.9)

* During the two periods cited in the table, letter of intent (LOI) or concepts were submitted to the National Cancer Institute (NCI) and subjected to a review process. LOI and concepts were reviewed and either approved or disapproved by either a Protocol Review Committee composed of NCI staff or by a more broadly representative disease-specific steering committee, respectively. Investigators could withdraw an LOI or concept before a review decision was made, and some LOI/concepts are still in review as of the time of this analysis. OEWG = Operational Efficiency Working Group.

Results

For the early-phase trials, Figure 2 demonstrates the reduction in median overall development time from 541 days pre-OEWG to 442 days post-OEWG, a 18.3% decrease. Although this difference in the medians is important, the most striking feature is the absence of a tail on the post-OEWG curve compared with the large number of trials with long development times in the bottom quartile in the pre-OEWG curve. A similar comparison of the phase III trials (Figure 3) is even more encouraging, with improvement in median time to activation from a median of 727 days pre-OEWG to 395 days post-OEWG, a 45.7 % decrease (n = 25). Importantly, none of the phase III trials exceeded or came close to the absolute deadline of 730 days in contrast with the pre-OEWG period where half the trials extended several years beyond 730 days (Figure 3).

Figure 2.

Comparison of pre–Operational Efficiency Working Group (OEWG) and post-OEWG timelines: early-phase studies. Kaplan–Meier plots of time from letter of intent (LOI) submission to trial activation (n = 525 pre-OEWG trials and n = 159 post-OEWG trials). Vertical solid line represents the absolute deadline, and the vertical dashed line represents the target timeline.

Figure 3.

Comparison of pre–Operational Efficiency Working Group (OEWG) and post-OEWG timelines: phase III studies. Kaplan–Meier plots of time from concept submission to trial activation (n = 112 pre-OEWG trials and n = 25 post-OEWG trials). Vertical solid line represents the absolute deadline, and the vertical dashed line represents the target timeline.

The pre- and post-OEWG timeline data for early-phase studies have been divided into three stages of protocol development and activation (Figure 4). When organized in this fashion, varying degrees of improvement are evident in all three stages of trial initiation. The time from LOI submission to LOI approval has been reduced by 29.1%, from a median of 110 days pre-OEWG to 78 days post-OEWG. The median time from LOI approval to protocol submission remained at 60 days in the pre- and post OEWG periods. However, there was much less deviation above the median for the post-OEWG trials at this stage. Improvements were also seen in the final stage encompassing protocol submission to trial activation. The median time decreased from 303 days pre-OEWG to 247 days post-OEWG, an 18.5% reduction. It is this final stage where the post-OEWG median number of days deviates the most from the target time of 90 days. Figure 5 separates this third stage into two components, protocol submission to protocol approval and protocol approval to trial activation. Although there has been a 28.4% improvement in the post-OEWG period compared with the pre-OEWG for the first component, the improvement is only 15.7% for the second component. As shown in Figure 1, this third stage of trial development, especially the second component from protocol approval to trial activation, encompasses a number of time-consuming steps, including local institutional scientific review, IRB approval, US Food and Drug Administration review, and contract negotiations. In addition, unanticipated drug supply issues can delay study activation.

Figure 4.

Early-phase trial development stage timelines pre–Operational Efficiency Working Group (OEWG) vs post-OEWG. Data shown as median days, with the bars representing the first and third quartiles of the distributions. For letter of intent (LOI) submission to LOI approval, there were 525 pre-OEWG trials and 159 post-OEWG trials with submitted LOIs. For LOI approval to protocol submission, there were 524 pre-OEWG trials and 152 post-OEWG trials with approved LOIs. For protocol submission to trial activation, there were 522 pre-OEWG trials and 143 post-OEWG trials with submitted protocols.

Figure 5.

Components of stage 3: protocol submission to protocol approval and protocol approval to protocol activation. Data shown are median days, with the bars representing the first and third quartiles of the distributions. For protocol submission to protocol approval, there were 522 pre–Operational Efficiency Working Group (OEWG) trials and 143 post-OEWG trials. For protocol approval to trial activation, there were 489 pre-OEWG trials and 109 post-OEWG trials. Protocol approval is defined as Cancer Therapy Evaluation Program approval of the protocol document.

Discussion

Two years after the implementation of the OEWG guidelines, initial results indicate a substantial reduction in the time required to activate early-phase trials and phase III trials. This is largely because of the implementation of the strict deadline, although other changes likely contribute. We believe an important improvement has been the joint NCI–investigator conference calls that occur 1 to 2 weeks after the review of both the LOI/concept and the protocol. These calls require that NCI staff complete their reviews rapidly to allow investigators sufficient time to prepare for the call. Despite the heavy time commitment involved, the calls have been well received by all parties because they provide an opportunity to immediately clarify divergent views.

The remaining challenge, however, is to achieve the OEWG target timelines rather than just meeting the absolute deadline. Although the first two stages of the process (NCI review of the LOI/concept and submission of the protocol by investigators) are “on average” close to their targets for the post-OEWG trials (Figure 4), the third stage, by contrast, has exceeded the 90-day target in every trial (data not shown). The principal explanations for delay in the third stage of protocol development include the time required for obtaining definitive pharmaceutical company commitments for investigational agents, sluggish contract negotiations, and prolonged IRB review timelines. The first two issues could actually be resolved much earlier in the protocol development process as drug commitment and contract negotiations are potentially amenable to resolution during the LOI/concept approval stage. Yet, many companies will not make a final commitment until they review the full protocol. Likewise, IRBs require a full protocol to conduct their evaluation, so other strategies are needed if IRB review timelines are to improve.

Although commercial sponsors have indicated that speed is a critical issue in their decision to partner with the NCI, obtaining company commitments to provide a supply of an investigational agent for proposed trials continues to be a source of delay in some cases. A potential solution might be to impose deadlines on this process. For example, NCI has instituted a 6-month deadline for the execution of all new Cooperative Research and Development Agreements (CRADAs) with companies wishing to partner with NCI. Because the CRADA defines the initial trials to be performed with NCI support, drug commitment should become more predictable for companies and will allow them to plan appropriately.

Because NCI is often unable to fully fund the cost of the research study, investigators seek additional support from companies. For many large phase II and III studies conducted by the cooperative groups and for some NCI-held IND early-phase studies, delays occur because of contract negotiations between the company sponsor and the investigator or group. This support is often for correlative science studies but may sometimes be for critical tests to monitor safety or response, such as additional radiologic tests, bioassays, or tumor collections. The time and effort involved in negotiating these additional sources of support is often responsible for major delays. Although NCI has set parameters (6) for intellectual property agreements related to its studies, the negotiations between foundations or universities and companies can be prolonged. Strategies to overcome this problem have been attempted by NCI and other organizations (7). One approach taken by NCI is to allow funding to be received under the CRADA to support these studies. The funds would then be allocated to the group or the site as a supplement or subcontract to the existing funding agreement. However, it remains to be seen if this approach alone will be successful. Other solutions are likely to be needed, including consideration of placement of a deadline on this negotiation step by groups and institutions similar to the approach taken for CRADA negotiations.

Delays related to IRB approval include tardiness in protocol submission to the IRB, excessive IRB time for review, and/or delays in responding to IRB comments. The use of NCI’s central IRB has reduced IRB review times for phase III trials as well as the cost to local institutions (8). Recent support by the Office of Human Research Protections for the use of central IRBs for multicenter trials (9) and support by others (10) may lead to increased central IRB use. This has led NCI to begin planning for an additional central IRB for multi-institutional phase I and II trials that would allow for faster activation of those studies.

Two years after implementation of the OEWG guidelines, it is noteworthy that only two early-phase trials and no phase III trials have been terminated in development because they exceeded the absolute deadline. For both of the disapproved trials, the delay in development was because of an unavailable supply of drug. This has persuaded both the NCI and the extramural investigator community that absolute deadlines for clinical trial activation can be decreased further to 450 days (15 months) for phase I–II trials and to 540 days (18 months) for phase III trials. This change was initiated April 1, 2012. Although activation is only the first step in completing a clinical trial, delays in activation have correlated with poor accrual and prolonged times to trial completion. Thus, improvements in trial initiation should encourage potential partners in the pharmaceutical and biotechnology industries to view the NCI’s publicly funded clinical trials system as a place where their new agent or device can be rapidly moved into the clinic.

Funding

This effort was supported by grant and contract funds from the National Cancer Institute, National Institutes of Health (U10CA076001 and U10CA031946 to Brigham and Women’s Hospital; U10CA098543 to Children’s Hospital of Philadelphia; U10CA021115 to Frontier Science & Tech Research Foundation, Inc; U10CA027469 to Gynecology Oncology Group; U10CA025224 to Mayo Clinic College of Medicine, Rochester; U10CA077202 to Queen’s University at Kingston; U10CA012027 to NSABP Foundation, Inc; U10CA021661 and U10CA080098 to American College of Radiology; U10CA032102 to University of Michigan at Ann Arbor OPS; U10CA070095 to Johns Hopkins University; U01CA099168 to University of Pittsburgh at Pittsburgh; U01CA132194 to University of Med/Dent NJ-R W Johnson Medical School; U01CA062502 to Case Western Reserve University; U01CA069912 and HHSN261201100099C to Mayo Clinic; U01CA07657 and HHSN261201100070C to Ohio State University; U01CA062490 to Dana-Farber Cancer Institute; U01CA062491 to University of Wisconsin–Madison; U01CA062487 to Wayne State University; U01CA062505 to City of Hope/Beckman Research Institute; U01CA069852 and HHSN261201100071C to University of Chicago; U01CA69856 to Sloan-Kettering Institute of Cancer Research; U01CA132123 to University Health Network; U01CA062461 and HHSN261201100039C to MD Anderson Cancer Center; HHSN261201100010C to Emmes Corporation; HHSN261201100032C to University Health Network, Princess Margaret Hospital; HHSN261201100038C to University of California at Davis; and HHSN261201100100C to H. Lee Moffitt Cancer Center and Research Institute).

The study sponsor participated in the design of the review; the collection, analysis, and interpretation of the data; the writing of the manuscript; and the decision to submit the manuscript for publication.

The authors would specifically like to express their appreciation to all of the members of the Operational Efficiency Working Group (2) who invested significant time and effort in developing the implementation plan that has been put into practice over the past 2 years.

This manuscript was previously published and presented at an ASCO meeting: Implementation of the Operational Efficiency Working Group (OEWG) Recommendations K. DiPiazza, E. Souhan, S. Finnigan, A. Denicoff, S. Friedman, M. Montello, M. M. Mooney, P. Schettino, J. A. Zwiebel, J. H. Doroshow, J. S. Abrams; EMMES Corporation, Rockville, MD and the Cancer Therapy Evaluation Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, Bethesda, MD, ASCO June 2011, general poster session.