Abstract
Purpose
The authors sought to assess the prevalence and associated economic impact of low-enrolling clinical studies at a single academic medical center.
Method
The authors examined all clinical studies receiving institutional review board (IRB) review between FY2006-FY2009 at Oregon Health & Science University (OHSU) for recruitment performance and analyzed them by type of IRB review (full-board, exempt, expedited), funding mechanism, and academic unit. A low-enrolling study included those with zero or one participant at the time of study termination. The authors calculated the costs associated with IRB review, financial set-up, contract negotiation, and department study start-up activities and the total economic impact on OHSU of low-enrolling studies for FY2009.
Results
A total of 837 clinical studies were terminated during the study period, 260 (31.1%) of which were low-enrolling. A greater proportion of low-enrolling studies were government-funded than industry-funded (P=.006). The authors found significant differences among the various academic units with respect to percentages of low-enrolling studies (from 10% to 67%). The uncompensated economic impact of low-enrolling studies was conservatively estimated to be nearly $1 million for FY2009.
Conclusions
A substantial proportion of clinical studies incurred high institutional and departmental expense but resulted in little scientific benefit. While a certain percentage of low-enrolling studies can be expected in any research organization, the overall number of such studies must be managed to reduce the aggregate costs of conducting research and to maximize research opportunities. Effective, proactive interventions are needed to address the prevalence and impact of low enrollment.
Academic medical centers (AMCs), or institutions with the core mission of conducting clinical research, play a vital role in the discovery of new knowledge, the evaluation of current therapies, and the education of the medical and local communities.1 One critical role of an AMC is to conduct research that advances basic scientific observations to applications in medical practice. The National Institutes of Health (NIH) has made translational research a priority, in part through the formation in 2006 of the Clinical and Translational Science Awards (CTSA) program. In that effort to improve bidirectional translational science, four strategic goals were formulated—the first of which is to enhance the national capability for clinical and translational research.2 This goal includes increasing efficiency, quality, and safety in the conduct of research.3 As one of the first CTSA-funded sites, Oregon Health & Science University (OHSU) has undertaken efforts to enhance translational science by identifying and overcoming local barriers to clinical research. One such important barrier is enrolling sufficient numbers of participants in clinical studies to support their stated scientific objectives.
The recognition of barriers to successful participant enrollment in clinical studies is not new—a study over 25 years ago found that one third of a cohort of 41 randomized clinical studies at the NIH recruited less than 75% of their planned enrollment goals.4 A sampling of 13 studies sponsored by the National Heart, Lung, and Blood Institute found that planned enrollment was completed for only two of the studies (15.4%),5 and more recent investigations of federally-sponsored oncology trials show that the proportion of studies that failed to achieve minimum recruitment goals ranged from over 25% to nearly 40%.6-8 Furthermore, specific milestones can be used early in the enrollment period to predict eventual enrollment success for oncology studies.9 Studies that do not achieve planned enrollments are unable to support their intended scientific hypotheses, thus reducing their scientific relevance and the efficiency of the entire clinical research enterprise.
To determine the institutional scope of this problem, to understand its impact on resource utilization, and to set the stage for effective remediation, we analyzed the prevalence of studies that were unsuccessful in enrolling participants and their associated administrative costs at a single AMC during a four-year period. We discuss the economic implications of such studies and present strategies for addressing this issue.
Method
Categorizing low-enrolling studies
We included all clinical studies at OHSU terminated between July 1, 2005 and June 30, 2009 (FY2006 – FY2009) in this analysis. We defined terminated studies as those in which participant recruitment and all research activities, including participant follow-up or data analysis, were completed during the study period. We collected data from electronic records maintained by the OHSU institutional review board (IRB). This database includes all OHSU human studies and study enrollment information at the time of each continuing review and at study termination. We defined low-enrolling studies as those with zero or one participant enrolled at the time of study termination. We chose this definition of low enrollment based on a threshold that would be considered a failure to meet scientific endpoint objects in almost every context. We included only terminated studies in the analysis, so we analyzed only confirmed final enrollment results. To better understand potential identifying characteristics of these studies, we examined low enrollment by (1) IRB review type, (2) academic unit conducting the research, and (3) funding mechanism.
IRB review type
We categorized clinical studies by IRB review type: full-board, expedited, and exempt. Studies requiring full-board review, as defined within the Code of Federal Regulations on IRB functions and operations,10 were approved at a convened meeting with a committee quorum. Expedited or exempt studies were reviewed and approved by the IRB chair or other qualified reviewers without involvement of the board. For the purpose of this analysis, we compared the prevalence of low-enrolling studies with full-board review to those with expedited/exempt review. We calculated the economic resources attributable to low-enrolling studies involving full-board review.
Academic unit conducting the research
We also assessed the prevalence of low-enrolling studies across academic units, including departments, divisions, or other organized research units. We limited analyses of academic units to those with more than 20 terminated studies between FY2006 to FY2009, which was the minimum sample size required to calculate median costs and avoid aberrant results.
Funding mechanism
Funding sources for clinical research include industry, government, and other sources. Industry-funded studies receive primary financial support from private corporations and include both those designed by industry and those developed by investigators at OHSU but supported by industry; the latter studies are typically termed investigator-initiated. Government funding includes federal- and state-supported studies. The other sources category includes those studies sponsored by agencies, such as foundations, or other institutions, those supported by multiple sources of funding, and those conducted without any extra-institutional financial support.
Calculating uncompensated costs due to low enrollment
We estimated uncompensated costs associated with low-enrolling, full-board reviewed studies for FY2009 across four primary sources of administrative support for the development and conduct of clinical research: (1) the principal investigators (PIs) and their academic units, (2) the Clinical Trials Office (CTO) contracting unit, (3) the OSHU IRB, and (4) the Sponsored Projects Administration (SPA).
PIs and their academic units
This source of administrative support provides substantial upfront investment in the development of a clinical study. Each academic unit requires specific and, to some extent, unique, processes and reviews. For example, the OHSU Knight Cancer Institute, a National Cancer Institute-designated cancer center, provides a separate scientific review of all oncology clinical studies. The PI and his or her staff must also prepare submissions to the IRB, negotiate and prepare study budgets, and implement the study, including collecting regulatory documents, interfacing with infrastructure units such as pharmacy and nursing, participating in investigator meetings and monitor visits, and creating recruitment plans.
CTO contracting unit
The CTO contracting unit is responsible for coordinating and negotiating contracts for industry-sponsored clinical studies. This negotiation process includes the review of confidential disclosure agreements (CDAs), clinical trial agreements (CTAs), subcontracts, and amendments.
OHSU IRB
The OHSU IRB reviews all research involving human subjects performed by OHSU faculty, research staff, and students. All open, non-exempt clinical studies are reviewed by the IRB at least annually, and any study design modifications or unanticipated problems are reviewed as necessary.
SPA
The SPA manages the financial aspects of all OHSU-sponsored projects. For each research award received by OHSU, the award account must be initiated, managed, and closed. Expenditures related to the conduct of clinical studies are managed through this office.
Calculating the cost of administrative support
We separated the cost of administrative support for study development into three types: (1) start-up, (2) maintenance, and (3) close-out costs (see Table 1). We included only full-board review studies in the cost analysis, as these types of studies are the most resource intensive. We also included only costs not directly reimbursable for efforts attributed to support low-enrolling studies.
Table 1.
Categories of Unreimbursed Costs Due to Low Enrollment of Clinical Studies, Oregon Health & Science University, FY2009
| Categories of cost | Explanation | Calculations included |
|---|---|---|
| Start-up costs | ||
| Clinical study preparation by study team | Costs associated with study team (excluding the principal investigator) in preparing industry- sponsored trials, including study tools, pre-initiation, and initiation meetings |
|
| Uncompensated industry clinical study start-up, budget, and institutional review board (IRB) preparation |
Costs not associated with Clinical Trials Office (CTO) support or other centralized unit |
|
| Non-industry study IRB full-board review | IRB costs associated with IRB reviewer’s efforts to conduct an IRB review and costs to support the review of a study |
|
| Confidential disclosure agreement (CDA) negotiation | Costs to conduct negotiations with the sponsor for industry-sponsored clinical studies |
|
| Clinical trial agreement (CTA) negotiation | Costs to conduct negotiations with the sponsor for industry-sponsored clinical studies |
|
| Award set-up | Costs require to set up study accounts |
|
| Maintenance costs | ||
| Continuing IRB reviews | Costs associated with preparation of continuing reviews mandated by regulations |
|
| Study modifications | Costs associated with study staff submission of modifications to a study during the enrollment period |
|
| CTA amendments negotiation | Number of amendments of existing agreements that are processed annually |
|
| Close-out costs | ||
| Award close-out | Costs required to close an account at the end of the study |
|
Start-up costs
Start-up costs are associated with activities required to prepare a clinical study for enrollment. Start-up costs include preparation of study materials, IRB initial review, preparation of study budgets, contract negotiation, awards set-up, and study planning meetings. Costs associated with study team preparation at the academic unit level for new studies presented to the IRB in FY2009 are based upon estimates of the average salary with fringe benefits for a study coordinator as well as previous estimations used by Dirvage and Bridges11 and C-Change.12 The salary for study coordinators, including fringe benefits, is estimated to be $60/hour. These costs are an underestimation of the total start-up costs, as they do not include the time and effort consumed by the study PI(s), biostatistician, or other study team members, such as co-investigators and collaborators, pharmacists, nursing staff, clinical department staff, and/or academic unit administrative staff. Some academic units have centralized resources to negotiate budgets and prepare studies for IRB review, and OHSU runs a centralized office that all academic units can utilize for these activities. These centralized resources recoup their costs through a non-refundable start-up fee payable upon completion of the activities. However, many studies at our institution do not utilize these central offices and do not recover their start-up costs unless they enroll participants.
To estimate IRB personnel effort, we included the anticipated time required to support a full-board review and approval process. In calculating IRB cost estimates, we excluded industry-designed studies, as the IRB charges review fees at initiation of those studies to recoup the cost of the review.
Industry sponsors often require CDAs prior to providing clinical trial documents. CDAs are reviewed by the CTO contracting unit to ensure the terms and conditions are in accordance with OHSU policies. Prior to the initiation of industry-sponsored clinical studies, the CTO contracting unit negotiates the terms and conditions of the CTA with the industry sponsor. The costs associated with low-enrolling studies reflect the estimated percentage of low-enrolling studies across the total number of CTAs and CDAs negotiated. Additionally, award set-up by SPA is required for all sponsored studies, and so the effort associated with setting up studies financially is attributed to all studies.
Maintenance costs
Maintenance costs are associated with the ongoing activities required to keep a study open regardless of whether participants are enrolled. For example, amendments to CTAs require review and negotiation by the CTO contracting unit, so costs are based upon estimated efforts. Also, the IRB must review non-exempt studies annually, as well as any modifications to the study. In our analysis, we used the number of IRB modifications and annual continuing reviews reported by the IRB during FY2009 and determined uncompensated costs for conducting these reviews accordingly.
Close-out costs
Close-out costs are generated when studies are terminated. Study termination must be reported to and reviewed by the IRB. In addition, the SPA must close the study account, involving review and reconciliation of all study charges.
Statistical analysis
We summarized categorical and ordinal groups using univariate and cross-tabulated frequency distributions. We compared low-enrolling studies for each identified characteristic using the Kruskal-Wallis test. We calculated post-hoc comparisons of statistically significant observations using the Mann-Whitney tests. We maintained a maximum two-tailed alpha of 0.05 for determining statistical significance. We performed all statistical analyses with SPSS (version 17.0, Chicago, IL, www.spss.com).
The OHSU IRB determined that this study did not require approval, as the data provided by the IRB did not include any identifiable information and were not obtained through interaction with human subjects.
Results
In our analysis, we found a total of 837 terminated studies conducted by 57 academic units between FY2006 and FY 2009, and we identified 260 (31.1%) as low-enrolling (see Table 2). This cohort of total terminated studies constituted 24.1% of all 3,470 clinical studies conducted at OHSU during the study period. Of the total sample of terminated studies, 383 (45.8%) required IRB full-board review, and, of these, 173 (45.2%) were low-enrolling studies. Of the 454 exempt or expedited studies (54.2% of all terminated studies), 87 (19.2%) resulted in low enrollment; significantly less than the 45.2% of full-board reviewed studies (P≤.001). Full-board studies included those involving therapeutic interventions and were generally more complex than expedited or exempt studies.
Table 2.
Terminated Clinical Studies by Enrollment and Institutional Review Board (IRB) Review Type, Oregon Health & Science University, FY2006-FY2009
| No. of studies (% of 837) |
No. of studies (% of review type total) |
||||
|---|---|---|---|---|---|
| ≤ 1 participant enrolled at termination |
>1 participant enrolled at termination |
Total no. of studies (%) |
≤ 1 participant enrolled at termination |
>1 participant enrolled at termination |
|
| IRB Review Type | |||||
| Full-board | 173 (20.7) | 210 (25.1) | 383 (45.8) | 173 (45.2) | 210 (54.8) |
| Exempt/Expedited | 87 (10.4) | 367 (43.8) | 454 (54.2) | 87 (19.2) | 367 (80.8) |
| Total | 260 (31.1) | 577 (68.9) | 837 (100) | – | – |
Comparison by funding mechanism
Table 3 summarizes the low enrollment rates by funding mechanism (industry, government, or other sources). The other sources category (such as internal, foundation, and multiple sources of funding) was the most common source of financial support for the studies that we examined (416 of 837 studies, 49.7%). Studies supported by other funding mechanisms had the smallest proportion of low enrollment (85 of 416, 20.4%; P≤.001). Clinical studies that received funding from government sources (97 of 837, 11.6%) had the highest incidence of low enrollment (52 of 97, 53.6%). Across all review types, government-funded studies had a greater proportion of low-enrolling studies compared to industry-funded studies (P=.006). Of those studies requiring full-board review, government-funded studies also had the greatest incidence of low enrollment (36 of 48, 75.0%) compared to studies supported by industry (110 of 271, 40.6%) and other sources (27 of 64, 42.2%) (government vs. industry: P>.001; government vs. other: P=.001). When comparing exempt/expedited studies by funding mechanisms, differences were observed between government (16 of 49, 32.7%) and other funding (58 of 352, 16.5%) (P=.006).
Table 3.
Terminated Low-Enrolling Clinical Studies by Funding Mechanism and Institutional Review Board Review Type, Oregon Health & Science University, FY2006-FY2009
| Full-board | Exempt / Expedited | All studies | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Funding mechanism |
Total studies |
Studies with low enrollment |
% with low enrollment |
Total Studies |
Studies with low enrollment |
% with low enrollment |
Total studies |
Studies with low enrollment |
% with low enrollment |
| Industry | 271 | 110 | 40.6 | 53 | 13 | 24.5 | 324 | 123 | 38.0 |
| Government | 48 | 36 | 75.0 | 49 | 16 | 32.7 | 97 | 52 | 53.6 |
| Other | 64 | 27 | 45.2 | 352 | 58 | 16.5 | 416 | 85 | 20.4 |
| Total | 383 | 173 | 45.2 | 454 | 87 | 19.2 | 837 | 260 | 31.1 |
Comparison across academic units
We identified 16 of the 57 (28.1%) academic units in the sample with more than 20 studies. These academic units accounted for 603 studies (72.0% of the total terminated studies). The number of low-enrolling studies within these academic units ranged from 29 studies (10%) to 46 studies (67%).
Economic impact of low-enrolling studies
To understand the economic implications (uncompensated costs), we calculated the estimated unit and administrative costs attributable to the low-enrolling studies. We estimated the uncompensated costs associated with studies terminated with low enrollments for study start-up, maintenance, and close-out activities across the involved infrastructure (IRB, CTO contracting unit, and SPA). A complete list of identified costs and the variables used for the cost estimations are found in Table 1. Our estimated financial impact of low enrollment is conservative, as it includes neither exempt nor expedited studies (454 of 837, 54.2%) nor PI time.
The total institutional uncompensated cost of studies that enrolled zero or one participant was approximately $990,000 in FY2009. Start-up and maintenance costs represented approximately $637, 000 (64.4% of the total cost) and $350,000 (35.4% of the total cost), respectively (see Table 4). Of the start-up costs, the largest proportion was associated with study preparation (about $466,000 or 47.1% of the total cost), including developing study tools, and pre-initiation and initiation meetings.12 We estimated the average cost to initiate one study to be $4,800. The largest sub-component of maintenance costs was associated with study modifications (about $315,000 or 31.8% of the total cost). We estimated the resources allocated to IRB review of modifications and continuing reviews of low enrolling studies to be $30,000 (3.1% of the total cost). We found close-out costs linked with low-enrolling studies (about $2,000 or .2% of the total cost) to be relatively small in proportion to both study start-up and annual maintenance costs.
Table 4.
Estimated Annual Economic Impact of Terminated Low-Enrolling Studies, Oregon Health & Science University, FY2009
| Activity | No. of low- enrolling studies requiring each activity |
Cost of activity ($) | Cost of activity for all low-enrolling studies ($) |
Total cost ($) (FY2009) |
|---|---|---|---|---|
| Start-up costs | 637,080 | |||
| Clinical study preparation by study team | 97 | 4,800 | 465,600 | |
| Uncompensated industry clinical study start-up, budget, and institutional review board (IRB) preparation |
20* | 3,250 | 65,000 | |
| Non-industry study IRB full-board review | 70 | 1,023 | 71,610 | |
| Confidential disclosure agreement (CDA) negotiation |
50 | 125 | 6,250 | |
| Clinical trial agreement (CTA) negotiation | 57 | 400 | 22,800 | |
| Award set-up | 97 | 60 | 5,820 | |
| Maintenance costs | 349,875 | |||
| Continuing IRB reviews | 229 | 130 | 29,770 | |
| Study modifications | 1614 | 195 | 314,730 | |
| CTA amendment negotiation | 43 | 125 | 5,375 | |
| Close-out costs | 1,940 | |||
| Award close-out | 97 | 20 | 1,940 | |
| Total: 988,895 |
A total of 45 full-board studies in FY2009 used central resources for start-up, budget, and/or IRB preparation and therefore charged non-refundable start-up fees to recoup the costs of these activities regardless of subsequent enrollment.
Discussion
Clinical and translational research in the United States has been characterized as fragmented, poorly coordinated, slow, and expensive.13-15 An important component of clinical research occurs at AMCs, but there are few objective assessments of its cost or effectiveness. Our findings focus on one major problem within the ailing system of clinical research: low-enrolling studies. Almost one out of every three terminated clinical studies at OSHU enrolled zero or one participant during the period of our research. Although our research was limited to a single AMC, it is highly likely that enrollment efforts and their costs are similar at other institutions; for example, research in oncology has shown a great deal of similarity in low-enrolling studies among different cancer centers.11 The costs of such low-enrolling studies are extensive and represent an important drain on research resources both locally and nationally, a particularly important issue in a constrained economic environment.
The financial implications of clinical studies that do not enroll enough participants are often overlooked and, while being relatively opaque to individual researchers, such costs can quickly accumulate to represent a significant amount of an institution’s resources. At our institution, the annual administrative cost for low-enrolling, full-board review studies was found to be almost $1 million for FY2009. Importantly, the consumption of these valuable resources presumably resulted in minimal scientific benefit. More worrisome, such studies indirectly may have prevented the conduct of other research that would have been more likely to achieve its primary endpoints through successful recruitment.
The scientific and financial implications of low enrollment warrant substantial efforts to foster awareness of their effects and to develop methods for prevention and mitigation. The largest proportion of cost was attributed to study start-up. Approaches to avoid this significant effort should include routine and objective feasibility assessments prior to supporting the initiation of a study. Ongoing maintenance costs were also substantial, suggesting a need for strategies to proactively recognize and close low-enrolling studies. Our research highlights three characteristics that might focus these efforts: (1) the type of IRB review required, (2) the funding mechanism, and (3) the type of academic unit conducting the research. Studies requiring full-board review and those funded by government sources more frequently involved low enrollment, hence an earlier review of such potential studies at the concept level could be helpful. Preemptively preventing studies that may present potential barriers to feasibility will avoid the allocation of the critical and significant resources necessary to start and maintain a study to research that is at-risk for low enrollment. As our data shows, interventions to close studies once they exhibit symptoms of low enrollment are suboptimal, as resources have already been spent and cannot be recovered. Finally, understanding and reducing the variation in study design and conduct among academic units could identify potential strategies directed to addressing low-enrolling studies. Our research highlights the overarching argument that implementation of improvements to clinical research cannot be done in isolation by any one group or one particular area of implementation; rather, there is a need to address barriers to clinical research from a concurrent and collaborative approach across all parties to create a more efficient and effective clinical research system, i.e., taking more of a systems approach.16
In addition, two specific cultural issues must be addressed to reduce the proportion of low-enrolling studies: the paradox of abundance and sunk-cost bias. The paradox of abundance is the overestimation of institutional resources potentially available to support the nearly infinite number of possible clinical studies. Management literature shows that, with such a misperception, there is an increased time to market (in our case, achievement of recruitment goals), an increased number of unexciting products (or studies), and an increased number of failures.17 There must be a widespread understanding that all groups operate within the boundaries of constrained resources and that the commitment of those resources must be carefully undertaken early in the process of study inception. The other cultural issue, sunk-cost bias, in which the irrecoverable costs incurred in building the study hinder rational decision-making in later phases, must be appreciated.18 It is well known that the level of effort and resources necessary to initiate clinical research is substantial; yet inappropriately continuing to pursue those studies that have a high likelihood of low enrollment incurs additional financial and resource costs. Recognizing the point at which low enrollment will not be reversed, and discontinuing efforts on the study, would restrain cost accumulation.
Our study has a few limitations. First, our research underestimated the true costs of low enrollment. We included only studies involving full-board IRB review and used a conservative estimate that incorporated only the standard institutional costs common across all clinical research. We also chose to include only terminated studies in our analysis and did not account for those studies that are ongoing yet continue not to enroll participants. In considering the total level of effort and financial implications of low enrollment, it must be recognized that each academic unit has unique approaches to the development and conduct of clinical research that will variably affect study costs, which we did not take into consideration. Moreover, our assessment of economic implications of low enrollment did not consider activities such as preparing and writing study protocols by the PI, completing study-specific training, and conducting recruitment activities. Clinical and administrative activities, including drug dispensing and maintenance, nursing support, regulatory reporting, and screening of participants, were also not included in our analysis. Finally, our analysis did not account for intangible costs, including the detrimental impact of low-enrolling clinical studies to the reputation of the university and PI, and relationships built across the research community.
Conclusions
Improving the productivity of research is one of the fundamental challenges to transforming clinical and translational science and cultivating a culture of excellence across all academic medical centers. The etiology of low-enrolling studies must be further examined to determine the underlying causes and thus enable prevention and mitigation. As part of a comprehensive effort to reform clinical research on an institutional level, we are developing an objective understanding of the characteristics and impact of low-enrolling studies. Our findings set the stage for undertaking targeted approaches to reduce the problem and increase clinical research effectiveness.
Ethical approval: The OSHU Institutional Review Board determined that this study did not require approval.
Acknowledgments
The authors would like to thank the Oregon Clinical and Translational Research Institute, the OHSU Institutional Review Board, and the Knight Cancer Institute for all their help and support for this study. Specifically, the authors wish to thank Susan Bankowski, JD, for providing most of the necessary data.
Funding/Support: This study was made possible with support from the Oregon Clinical and Translational Research Institute, the National Center for Research Resources (grant number UL1 RR024140), a component of the NIH, and the NIH Roadmap for Medical Research.
Footnotes
Other disclosures: None.
Previous presentations: The results of this study were presented in part at the 2010 Cancer Trial Accrual Symposium and the 2010 Annual CTSA Clinical Research Management Workshop, both held in Bethesda, Maryland.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Darlene R. Kitterman, Ms. Kitterman is the director, Investigator Support and Integration Services, Oregon Clinical and Translational Research Institute, Oregon Health & Science University, Portland, Oregon.
Steven K. Cheng, Dr. Cheng is senior research associate, Knight Cancer Institute, Oregon Health & Science University, Portland, Oregon.
David M. Dilts, Dr. Dilts is director of clinical research, Knight Cancer Institute, and professor of management, Division of Management, Oregon Health & Science University, Portland, Oregon.
Eric S. Orwoll, Dr. Orwoll is director, Oregon Clinical and Translational Research Institute, Oregon Health & Science University, Portland, Oregon.
References
- 1.Kassirer JP. Academic medical centers under siege. N Engl J Med. 1994;331:1370–1. doi: 10.1056/NEJM199411173312010. [DOI] [PubMed] [Google Scholar]
- 2.Reis SE, Berglund L, Bernard GR, Califf RM, Fitzgerald GA, Johnson PC. [Accessed July 13, 2011];Reengineering the national clinical and translational research enterprise: the strategic plan of the National Clinical and Translational Science Awards Consortium. Acad Med. 2010 85:463–9. doi: 10.1097/ACM.0b013e3181ccc877. Available at: http://journals.lww.com/academicmedicine/Fulltext/2010/03000/Reengineering_the_National_Clinical_and.20.aspx. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Pober JS, Neuhauser CS, Pober JM. Obstacles facing translational research in academic medical centers. FASEB J. 2001;15:2303–13. doi: 10.1096/fj.01-0540lsf. [DOI] [PubMed] [Google Scholar]
- 4.Charlson ME, Horwitz RI. Applying results of randomised trials to clinical practice: impact of losses before randomisation. Br Med J (Clin Res Ed) 1984;289:1281–4. doi: 10.1136/bmj.289.6454.1281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Shea S, Bigger JT, Jr., Campion J, et al. Enrollment in clinical trials: institutional factors affecting enrollment in the cardiac arrhythmia suppression trial (CAST) Control Clin Trials. 1992;13:466–86. doi: 10.1016/0197-2456(92)90204-d. [DOI] [PubMed] [Google Scholar]
- 6.Cheng SK, Dietrich MS, Dilts DM. A Sense of Urgency: Evaluating the Link between Clinical Trial Development Time and the Accrual Performance of Cancer Therapy Evaluation Program (NCI-CTEP) Sponsored Studies. Clinical Cancer Research. 2010;16:5557. doi: 10.1158/1078-0432.CCR-10-0133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Go R, Meyer C, Mathiason M, et al. Nature and outcome of clinical trials conducted by the Eastern Cooperative Group (ECOG) from 1977 to 2006. American Society of Clinical Oncology; Chicago, IL: 2010. p. 6069. 2010. [Google Scholar]
- 8.Schroen AT, Petroni GR. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clinical Trials. 2010;7:312. doi: 10.1177/1740774510374973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cheng SK, Dietrich MS, Dilts DM. Predicting Accrual Achievement: Monitoring Accrual Milestones of NCI-CTEP-Sponsored Clinical Trials. Clinical Cancer Research. 2011;17:1947. doi: 10.1158/1078-0432.CCR-10-1730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.US Department of Health and Human Services [Accessed July 27, 2011];Protection of human subjects. 45CFR46.108. http://edocket.access.gpo.gov/cfr_2003/octqtr/45cfr46.108.htm. Revised October 1, 2003.
- 11.Durivage H, Bridges K. Clinical Trial Metrics: Protocol Performance and Resource Utilization from 14 Cancer Centers. J Clin Onc. 2009;27(suppl) abstr 6557. [Google Scholar]
- 12.C-Change . A Guidance Document for Implementing Effective Cancer Clinical Trials. Ver 1.2 ed. C-Change; Washington, DC: 2005. Executive Summary. [Google Scholar]
- 13.Califf RM, Berglund L. [Accessed July 13, 2011];Linking scientific discovery and better health for the nation: the first three years of the NIH’s Clinical and Translational Science Awards. Acad Med. 2010 85:457–62. doi: 10.1097/ACM.0b013e3181ccb74d. Available at: http://journals.lww.com/academicmedicine/Fulltext/2010/03000/Linking_Scientific_Discovery_and_Better_Health_for.19.aspx. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dilts D. US cancer trials may go the way of the Oldsmobile. Nat Med. 2010;16:632. doi: 10.1038/nm0610-632. [DOI] [PubMed] [Google Scholar]
- 15.Dilts D. Phase III Clincial Trial Development: A Process of Chutes & Ladders. Clin Cancer Res. 2010:16. doi: 10.1158/1078-0432.CCR-10-1273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dilts DM, Cheng SK, Crites JS, Sandler AB, Doroshow JH. Phase III clinical trial development: a process of chutes and ladders. Clinical Cancer Research. 2010;16:5381. doi: 10.1158/1078-0432.CCR-10-1273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Cooper RG, Edgett SJ, Kleinschmidt EJ. Portfolio management for new products. 2nd ed. Perseus Pub.; Cambridge, MA: 2001. [Google Scholar]
- 18.Staw B. Knee Deep in the Big Muddy. Organizational Behavior and Human Decision Process. 1976;35:124–40. [Google Scholar]