Development of Clinical Trials in a Cooperative Group Setting: The Eastern Cooperative Oncology Group

Abstract

PURPOSE

We examine the processes and document the calendar time required to activate Phase II and III clinical trials by an oncology group: the Eastern Cooperative Oncology Group (ECOG).

METHODS

Setup steps were documented by: 1) interviewing ECOG headquarters and statistical center staff, and committee chairs, 2) reviewing standard operating procedure manuals, and 3) inspecting study records, documents, and emails to identify additional steps. Calendar time was collected for each major process for each study in this set.

RESULTS

Twenty-eight Phase III studies were activated by ECOG during the 01/2000–07/2006 study period. We examined in detail a sample of 16 of those studies. More than 481 distinct processes were required for study activation: 420 working steps, 61 major decision points, 26 processing loops, and 13 stopping points. Median calendar days to activate a trial in the Phase III subset was 783 days (median, 285 to 1542 days) from executive approval and 808 days (range, 435 to 1604 days) from initial conception of the study. Data were collected for all Phase II and Phase III trials activated and completed during this time period (n=52) for which development time represented 43.9% and 54.1% of the total trial time respectively.

CONCLUSION

The steps required to develop and activate a clinical trial may require as much or more time than the actual completion of a trial. The data demonstrates that to improve the activation process, research should to be directed toward streamlining both internal and external groups and processes.

BACKGROUND

There is little question that streamlining the process of activating cancer clinical trials would be beneficial. Unfortunately, the majority of research being done in this area deals with post-activation efficiency and effectiveness. Previous research evaluating an oncology cooperative group (Cancer and Leukemia Group B, CALGB) documented that the clinical trial development process is highly complex, with numerous processing steps, decision points and processing loops (1). Questions have been raised as whether the results were unique to that specific setting or were more pervasive to other major oncology groups. Pursuant to that we have completed an analysis of the Eastern Cooperative Oncology Group (ECOG).

There are four types of administrative or setup barriers to activation processes at cooperative oncology groups: procedural, structural, infrastructural, and synchronicity. The first three of these types of barriers are common among cooperative oncology groups, comprehensive cancer centers, and community oncology practices (2). While synchronicity is an issue for any clinical trial, it is exacerbated in cooperative groups as a result of dealing with multiple external agencies (1).

Procedural barriers are policies (formal or informal) that are inherent to the process, or series of steps necessary to activate a study, and that may inhibit other problem-solving actions. For example, a procedural delay for a cooperative oncology group occurs when, after a concept has been internally approved, that concept must then be reviewed by an outside agency before investigators at ECOG can continue to develop the concept. Our findings show that such procedural barriers occur throughout the activation process and do so both within ECOG and at the external interface with other participants in the activation process such as the National Cancer Institute (NCI; National Institutes of Health, Bethesda, MD), the NCI Cancer Therapy Evaluation Program (CTEP), the NCI Central Institutional Review Board (CIRB), the US Food and Drug Administration (FDA), with pharmaceutical firms, and with trial study chairs, who volunteer their services while typically holding positions at universities and academic medical centers.

The second type of barrier, structural (i.e., barriers created by the design of the organization or the interface between organizations), arises typically whenever different participants in the process follow a different ordering of steps, often leading to miscommunication and misunderstanding. This barrier is represented by the circular mismatch loop that arises when one participant sends information to another, who then replies back to the sender—a correspondence that must take place before the process can proceed. For example, study chairs and cooperative groups run on different timelines with respect to responding to changes. This can lead to a circular, often protracted, situation: one group cannot collect the required information until it approves a condition, but it will not approve the condition without the required information.

Infrastructural barriers are those barriers that arise from the design of the underlying system and the interconnection between various system aspects. Consider the CTEP infrastructure as an example: protocol review, drug distribution, NCI Common Data Element (CDE) and CIRB review are each a basic part of the required infrastructure (3, 4). Each of these aspects is individually productive but can cause delays in activation as studies wait for final approvals from multiple parts of the system—especially because each aspect may function according to different timelines, support system, or oversight.

The final type of barrier, synchronicity, is characterized as the need for compilation of various components of a concept or a protocol before it can be submitted to other participants in the process. In other words, synchronicity results from a myriad of steps having to come together prior to a study advancing toward activation. At ECOG, for example, protocol assembly, financial affairs, forms/data management, and drug acquisition/distribution—each handled by a different functional group—all must be completed and compiled or synchronized before a study can be activated.

Although these are the four main types of hurdles faced by cooperative oncology groups, each can be exacerbated by simultaneous occurrence with other barriers. This kind of interactive effect can be observed in the overlap of administrative and synchronicity barriers which occur whenever CTEP, pharmaceutical company, FDA and CIRB reviews are completed sequentially. External reviews may be fragmented and sometimes contradictory because they collect information at various times from various group or individuals. Thus, these represent both timing and synchronicity barriers.

METHODS

Study Settings and Timeframe

The setting for this research is the Eastern Cooperative Oncology Group (ECOG), a national clinical research group sponsored by the National Cancer Institute (NCI), with its Group Chair’s office located in Philadelphia, Pennsylvania and the Coordinating Center located in Boston, Massachusetts. ECOG was established in 1955 as one of the first cooperative groups launched to perform multi-center cancer clinical trials. A cooperative group is a network of researchers, physicians, and health care professionals at public and private institutions across the country who are members of the group. Funded primarily by the NCI, ECOG has evolved from a five member consortium of institutions on the East Coast to one of the largest clinical cancer research organizations in the U.S. with almost 6000 physicians, nurses, pharmacists, statisticians, and clinical research associates (CRAs) from the U.S., Canada, Australia, Peru, Israel and South Africa. Institutional members include universities, medical centers, Community Clinical Oncology Programs (CCOPs), and Cooperative Group Outreach Programs (CGOPs). These institutions work toward the common goal of controlling, effectively treating, and ultimately curing cancer (5).

A sample of ECOG Phase III studies activated in a 6-year period (January, 2000 through July, 2006) was investigated (n=16). Sixteen studies were selected to provide a sufficiently broad set of studies. Phase III studies were chosen because they are the most complex, i.e., they contain all the relevant steps in study activation needed for all types of treatment trials, and because other research has shown that there may not be significant difference in calendar time to open by phase (2).

Because studies activated in year 2000 may had begun the process much earlier, the actual scope of time investigated was a period of 8 years (February, 1998 through July, 2006) to acquire all relevant timing data. All trials investigated were therapeutic studies; intergroup studies were excluded so as to concentrate on identifying where ECOG could reduce internal process steps and times.

To study development versus operation time in a clinical trial, data were collected for all ECOG Phase II and III therapeutic studies activated in the study period. One of the sixteen Phase III studies was still open to accrual during the study period and was excluded from this part of the analysis hence the sample size for this portion of the research fifteen.

Part A: Process Mapping

There are two primary aspects of the Dilts and Sandler method (2) used: process mapping and timing analysis. Process mapping consists in a graphical representation of the flow of inputs, resources, steps, and processes (both work and decision) required to create an output—in this case, to activate a study (6).

To generate the process map, a team of experts from the Center for Management Research in Healthcare (cMRHc) was engaged. Individuals from the Vanderbilt-Ingram Cancer Center (Nashville, TN), the Vanderbilt University (Nashville) School of Engineering (VUSE), and the Vanderbilt University Owen Graduate School of Management (OGSM) comprise the cMRHc. Data were collected by means of initial onsite personnel interviews, additional follow-up e-mail correspondence, and a series of clarification teleconferences with members of ECOG. Using these information sources, the team developed initial process flows needed for all major activities required to activate a Phase III study, including concept development, protocol development, forms and data management, financial affairs (if applicable) and regulatory affairs. Additional onsite clarification interview sessions were conducted prior to the final results presentation to verify the flows with relevant participants and to validate the steps shown by consulting individual study documentation.

Part B: Timing Analysis

For this aspect of the method, the calendar time needed for each of the major processes required to activate a study was collected. It is important to note that work time (i.e., the amount of effort in writing a study) was not evaluated, but that attention was focused only on calendar time (i.e., the days from initiation to completion of a process step). This data were contained in various forms, including written documentation, formal electronic formats, and informal e-mail communication.

Part C: Detailed Analysis of Two Studies

In addition to process mapping and timing data analysis for the full sample of studies, detailed analysis was conducted for two studies that represented a relatively “fast” and “slow” activation process, respectively. For both studies, the calendar time the concept or protocol remained at various elements of the system were identified and tracked, i.e., when the study with the study chair, with ECOG, with CTEP and with CIRB. For the slower study, a detailed process map was created showing explicitly the number of process loops and timing data for each step of the process. A similar analytical tool was created for the faster of the two case studies, showing time between each step as well as overall time taken to activate.

RESULTS

Part A: Process Mapping

Process maps can be created by (1) listing all process steps on one large complete diagram, or (2) creating a hierarchy of diagrams, each of which provides additional detail from a master, or Level 0, diagram (2, 7). Because of its size (605″ × 60″ in 8 point font), the complete process diagram is impossible to reproduce here. Figure 1 presents an overview a Level 0 diagram¹.

Figure 1.

Level 0 process flow map for activating a phase III study at Eastern Cooperative Oncology Group (ECOG). Days are the median calendar days from receipt to acceptance by the process. Notes: Concept Development refers to the time the concept was developed by the PI when data was available (n=5). Brown boxes represent joint efforts between ECOG and external governmental agencies. Grant development is necessary only if additional funding was requested from a pharmaceutical firm; Regulatory Affairs Development is necessary only if the source of the IND is not held by the NCI or if commercial agents are only used for the study. While initial FDA review is within 30 – 45 days, studies required multiple loops to attain final FDA approval. Abbreviations: CTEP, National Cancer Institute Cancer Therapy Evaluation Program; CIRB, National Cancer Institute Centralized Institutional Review Board; CDE, National Cancer Institute Common Data Elements; FDA, US Food and Drug Administration.

There are thirteen aspects to study activation at the ECOG:

1. Concept development by member both inside and external to ECOG

2. ECOG executive review,

3. Concept consensus review by an external agency,

4. Protocol development,

5. Protocol consensus review by an outside agency,

6. Forms development and data management development,

7. Common Data Element (CDE) compliance review by an outside agency,

8. Grant Development, which includes internal and external groups to ECOG (i.e., the NCI and/or Pharmaceutical firms),

9. Regulatory Affairs,

10. FDA review,

11. Centralized Institutional Review Board (CIRB) review,

12. Final external approval, and

13. Study activation activities.

At the initial stages, concept development and executive review, a study chair presents a potential Phase III concept study to an ECOG disease or modality committee chair. It is important to note that these individuals, while members of ECOG, are volunteers in the process. Each concept is discussed with the committee chair and, if sufficient time, the idea is presented before the disease or modality committee. If approved by the committee, the concept is presented for ECOG executive committee concept review. If the executive committee sends the concept back to for revision and resubmission, the first of 26 processing loops occurs. A loop is any time the concept or protocol is returned to a previous process step for change. If the concept is approved by the executive committee, it is sent to CTEP for concept review and approval. Frequently, this review process may require an additional loop in the process.

Once approved by the executive committee, the ECOG study team begins work on the study protocol. Protocol development is the most resource- and time-intensive aspect of the activation process, involving multiple process steps which include interaction between the study chair, ECOG and CTEP. The process of protocol development begins with efforts by the study chair and ECOG protocol developers to craft the details of the protocol from the concept. Once a protocol is sufficiently advanced, it is sent to CTEP for a protocol consensus review. As with concept review, this may initiate a series of process loops between the study chair, ECOG and CTEP due to requested changes. Such changes can be minor (addressed in a local process loop) or major (requiring the study to return to an earlier step in the process, potentially returning to additional executive and concept review). This aspect of the overall process continues until the protocol is conditionally approved by CTEP pending CIRB review.

Concurrent to the protocol development and review processes are the steps required for forms and data management. These processes begin within ECOG after the initial protocol has been sufficiently developed that forms can be created. Once the forms are completed within ECOG, they are submitted for NCI common data element (CDE) compliance review. Process loops also occur during these processes

If additional funding is needed for a particular study, ECOG begins grant development or contract negotiations with a potential sponsor immediately upon concept acceptance by the executive committee. It is important to note that, while grant/contract development appears to take extensive time, in most cases the final contract must await the final study budget, which is dependent upon the final study protocol. Hence, the majority of effort may be completed on the budget and contract early but the “clock” continues to “tick” because the final statement of work and contract has not been agreed upon and signed.

The final major process flow that runs concurrently with protocol development is regulatory affairs and drug acquisition (if necessary). This identifies the source of the drug to be studied, establishes a drug distribution plan, and determines Investigational New Drug (IND) requirements. Once these aspects are completed, the IND application (if required) is sent for FDA review.

Once the protocol has been accepted, the CIRB commences its review. Again, this process may require looping to early steps in the process. With the final CIRB approval, there is one final CTEP approval (the “final-final” review) before the last activation activities can take place.

One often overlooked feature of the process and timing requirements of these steps is the need for synchronicity. This is the idea that various elements of the protocol need to merge at the proper time for the study to be approved by internal and external organizations. For illustration, compare a similar phenomenon in the automotive industry: the correct engine must be matched with the correct body in the assembly line before any additional work can be done on the vehicle. If the correct engine is not ready, the assembly process stops until the engine is ready and the line restarts. In study activation, synchronicity is required between protocol development and forms/data management before the study can be sent for CIRB review. Additionally, all processes (i.e., CTEP, CIRB, FDA, and sponsor negotiations) must be synchronized prior to activation. This means that there are several potential areas of developmental lag, created when one group or step in the process is held up because a prior step has not yet been completed.

Although informative for overall flow of processes, the Level 0 process map does not indicate the number of steps, decision points, processing loops or stopping points. The ECOG Phase III study trial process requires at least 481 processing steps composed of:

• At least 420 working steps

• 61 major decision points

• 26 processing loops

• 13 stopping points (9 external to ECOG and 4 internal)

As a loop may return a protocol for reprocessing, the 481 steps identified represent the minimum set of steps possible. In actuality, due to the high likelihood of looping, the actual number for a study can be considerably greater. Examples of these loops are discussed in Part C.

Part B: Timing Analysis

During the comprehensive chart review, timing data were collected on 16 sample Phase III studies. To ensure accuracy of data collected, data were checked for completeness and consistency by cross-referencing electronic records with paper records. Those studies that had missing dates were not included in summary timing analysis shown in Table 1.

Table 1.

Calendar Days From Initiation to Final Approval for Each Major Process Step Required to Activate a Phase III Study at the Eastern Cooperative Oncology Group

	No.	Median (days)	Range (days)	Number of Revision Loops	Range of Loops
Concept Development
Includes Study Chair Effort	5	72	37–95
…Includes only ECOG Effort	11	18	10–62
Executive Review	13	11	5–45	1	1
CTEP Concept Review	15	113	20–611	9	1–3
Protocol development	16	439	57–968
CTEP protocol review*	16	129	5–783	13	1–4
Forms/data management	15	401	50–1320
CDE compliance*	6	114	4–593	3	1–3
Grant development	11	750	294–993	1	3
Regulatory affairs development	1	102	102
FDA review*	1	46	46
CIRB review *	14	116	47–739	12	1–9
CTEP Final Approval	13	14	2–134
Preactivation Activities	1	42	42
Activation	13	808	435–1604

NOTE: The reasons for unequal sample sizes are as follows: in concept development, not all records were maintained by committee chairs to include study chair effort time; CDE, occurred midway through the study period, hence not all studies required this step; grant development, captured if only additional funding was requested from a pharmaceutical firm; regulatory affairs, completed only if the source of a drug is being received directly from a company; FDA review, necessary only if the IND is not held by NCI or the industry sponsor; While initial FDA review is within 30 to 45 days, studies required multiple loops to attain final FDA approval.

Abbreviations: CTEP, Cancer Therapy Evaluation Program; CDE, Common data elements; CIRB, Centralized Intuitional Review Board; FDA, US Food and Drug Administration; ECOG, Eastern Cooperative Oncology Group; IND, investigational new drug; CTEP, Clinical Trials Evaluation Program, NCI, National Cancer Institute, National Institutes of Health, Bethesda, MD.

^*Processes external to ECOG

Days do not sum as process steps overlap

For the sample studies, the median calendar time from initial concept proposal by a study chair to ECOG to study activation by ECOG was 808 calendar days (n=13; range, 435 to 1604 days. Median time from formal approval of the concept by the ECOG executive committee to study activation was 783 days (n=14; range, 285 to 1542 days). Only one study analyzed had dates for regulatory affairs, and noted 800 days to complete. The longest single process with a representative sample was grant development (median, 750 days; range, 294 to 993 days); however recall that the grant process must await final protocol approval before it can be completed. Not surprisingly, protocol development took the next longest (median, 439 days; range, 57 to 968 days), with forms/data management taking only 38 days less (median, 401 days; range, 50 to 1320 days).

For further analysis, we expanded the timing research to include additional sets of timing data. First, the time from executive committee approval to activation for all Phase III therapeutic studies with concept submission and activation dates from 1/2000 to 7/2006 was collected. For this data, the median time was 755.5 (n=28; range, 330 to 2801). This helps assure us that the studies sampled in detail are representative.

To investigate the question of how much time does concept and protocol development require of the total time to complete an oncology clinical trial, data were collected for all ECOG Phase II and III therapeutic studies activated between1/2000 and 7/2006 and closed to accrual by 7/2006. A total of 52 studies were identified. For Phase II studies, 43.9% of total study time (development plus operation) was spent in activities required to activate the study (n=37, development median time=562 days, operation median time=717 days) (See Figure 2). For Phase III trial, development time exceed operation time it required 54.1% of total study time (n=15, development median=877, operation median=743). With development time requiring such a large percentage of total study time, further investigation is required in the important area of research into the development of oncology clinical trials.

Figure 2.

Median Number of Development and Study Open Days for Phase II and III ECOG Clinical Trials opened for accrual and closed from 1/2000 to 7/2006

As one final investigation of time, the median development times for all Phase II and III studies activated between 2000 and 2006 were analyzed (See Figure 3). While no general trend observations can be made, the high variance in times to activate a trial shows that it is difficult to accurately predict when a study may be activated. Again, this demonstrates the need for additional research in this aspect of oncology clinical trials.

Figure 3.

Median days for development for therapeutic studies activated from 1/2000 to 7/2006. Figure 3a are Phase II studies (n=71) and Figure 3b are Phase III studies (n=28).

Part C: Detailed Analysis of Two Studies

In order to investigate the looping aspects and to potentially identify the rate limiting process, two Phase III studies were identified that had the most complete record of processing dates. The following timing definitions were used:

1. Concept development and review: the time from ECOG’ s initial receipt of a concept to CTEP concept approval.

2. Initial protocol development: the time from CTEP concept approval to initial protocol submission to CTEP.²

3. Protocol Review: the time from initial protocol submission to CTEP to CTEP approval pending CIRB approval.

4. CIRB review: the time from CTEP approval pending CIRB to CIRB approval of the protocol.

5. CTEP final review and study activation: time from CIRB approval of the protocol to study activation.

6. Total Time is the calendar days from ECOG’s initial receipt of a concept to study activation.

The faster of the two studies required a total time of 599 calendar days from receipt of concept to study activation (see Table 2). Within that time period, there were a total of 8 reviews conducted:

1. Concept development and review: 281 days, one executive, one initial CTEP and one revision review

2. Initial protocol development: 71 days³, no revisions

3. Protocol Review: 62 days, one initial and one revision review

4. CIRB review: 133 days; one initial and one revision review

5. CTEP final review and study activation: 52 days, one review.

With respect to the calendar time spent at each location or organization: ECOG/Study Chair accounted for 346 of the days (not counting additional time spent on Protocol Development); CTEP, 162 days; and CIRB, 91 days. It is interesting to note that only one revision review was required at each step of the activation process for this particular study.

Table 2.

Detail of Calendar Days Per Entity/Organization and Process Segment for two Phase III Studies Activated by the Eastern Cooperative Oncology Group

	Concept Development and Review	Initial Protocol Development*	Protocol Review	CIRB Review	CTEP Final Review and Study Activation	Total
Initial Time Point	Initial ECOG Receipt	CTEP Concept Approval	Initial CTEP Protocol Submission	CTEP Approval Pending CIRB	CIRB Approval	Initial ECOG Receipt
Final Time Point	CTEP Concept Approval	Initial CTEP Protocol Submission	CTEP Approval Pending CIRB	CIRB Approval	Study Activation	Study Activation
Fast Study
ECOG/Study
Chair	172	71	19	42	42	346
CTEP	109		43		10	162
CIRB				91		91
Total	281	71	62	133	52	599
Slow Study
Study Chair	49		27	95		171
ECOG	59	189	2	149		399
CTEP	98		55	127	2	282
CIRB				123		123
Total	206	189	84	494	2	975

^*Total actual protocol development time was 331 days (for the fast study) and 339 days (for the slow study).

These totals include the days listed above plus days from Executive Approval through the end of CTEP Concept Approval (260 and 150 days, respectively).

The slower study required 17 reviews and a calendar time of 975 days from executive committee receipt to study activation:

1. Concept development and review: 206 days, one executive, one initial CTEP and one revision review

2. Initial protocol development: 189 days⁴, no revisions

3. Protocol Review: 84 days, one initial and one revision review

4. CIRB review: 494 days; one initial and four revision reviews

5. CTEP final review and study activation: 2 days, one review.

In addition to the above 11 reviews, there were an additional six reviews triggered from preactivation amendments. Broken down by individual entity or organization, Study Chair time was 171 days, ECOG time was 399 days (not counting additional time spent on Protocol Development), CTEP time was 282 days, and CIRB time was 123 days.

Clearly one significant difference between the slow and the fast study is the number of revisions required (8 versus 17). Again, it should be mentioned that these are calendar days and not actual working days. It should also be noted that in some instances, responsibility for longer time includes the initiator of an activity. In other words, it may take one entity or agency only 3 days to make a decision that another entity or agency must engage in an activity that takes 60 days to complete.

DISCUSSION

Our focus is on assessing the barriers that can impede the activation of a clinical trial study at a major NCI-sponsored cooperative oncology group, ECOG. Given that, with more than 481 process steps, it is clear that the process for activating a study at the ECOG is highly complex and involves input from multiple individuals and agencies. As shown in the detailed flow of one Phase III trial, a study must pass through multiple hands, multiple times before gaining final acceptance for activation. As has been shown in numerous industries, the more frequently work is handed-off, the greater the likelihood for error and adding additional inspection points only exacerbates the problem.

The results presented here are consistent with research which analyzed the development process of another NCI-sponsored cooperative group, the CALGB (1). Table 3a presents various process count information concerning work steps, decision points, processing loops and stopping points for ECOG and CALGB respectively. What is most important to clinical research is time to study activation; comparison data for time to activate Phase III clinical trials at ECOG and CALGB are shown Table 3b. Note the similarity between the median time for both groups and the broad range between the minimum and maximum time for both cooperative groups.

Table 3.

Table 3a Comparison of Process Counts Between ECOG and CALGB
	ECOG	CALGB
Processing Steps	>481	>370
Work Step	>420	>328
Decision Points	61	42
Processing Loop	26	29
Stopping Points	13	16

Table 3b Phase III Activation Timing Comparison of ECOG and CALGB
	ECOG	CALGB
n	28	13
Median (days)*	808	784
Minimum (days)	435	537
Maximum (days)	1604	1130

Note: Values are calendar days

Abbreviations: ECOG Eastern Cooperative Oncology Group; CALGB Cancer and Leukemia Group B

In addition to the similarities in the data there are common structural issues intrinsic to the systems: the work of the investigators in the field who conceive of and develop the protocols is largely voluntary; the cooperative groups do not control, nor are they responsible for, many crucial steps in the process; and there is a layering of bureaucracies within the groups, the NCI, other government agencies and the pharmaceutical industry. In spite of these difficulties and inefficiencies the NCI-sponsored cooperative group system continues to produce significant therapeutic advances for both adult and pediatric cancer patients, as noted by the American Society of Clinical Oncology (8, 9).

One of the most striking findings in our current analysis relates to the discovery that the development time equals or exceeds the time required to complete the accrual of patients onto a study. This circumstance delays the time to the development of new treatments for cancer patients. Accordingly, the issues raised here must be addressed in order to speed the time to the decision making point on the benefits, or lack thereof, of a specific clinical trial---by all parties involved in the process.

Our findings indicate that a multi-pronged approach is necessary to decrease calendar time for activating Phase III trials. First, there are internal process changes that should occur within the cooperative groups. With the all the processes identified in a process map, it is possible to determine those steps that do not add value to the resulting protocol. Such steps should be eliminated. Second, it is important to establish study selection guidelines that include study operational complexity in addition to scientific merit. Third, it is important to acknowledge that even simple changes to a concept or protocol may require an extensive looping process thus greatly adding to time to activation.

It is important to note that improving processes solely within cooperative groups, without addressing their need to interface with external organizations, may result in minimal impact on the time to activate a phase III study. Not only is it important to streamline the intramural processes at cooperative oncology groups, but it is also essential that inter-agency processes be made more efficient. Currently, we are in the process of investigating such inter-agency interactions.