Abstract
OBJECTIVES:
Clinical trials are pillars of modern clinical evidence generation. However, the clinical trial enterprise can be inefficient, and trials often fail before their planned endpoint is reached. We sought to estimate how often urologic oncology trials fail, why trials fail, and associations with trial failure.
METHODS:
We queried phase 2/3 urologic clinical trial data from ClinicalTrials.gov registered between 2007–2019, with status marked as active, completed, or terminated. We extracted relevant trial data, including anticipated and actual accrual, from trial records and ClinicalTrials.gov archives. We manually coded reasons given in the “why stopped” free text field for trial failure into categories (poor accrual, interim results, toxicity/adverse events, study agent unavailable, canceled by the sponsor, inadequate budget, logistics, trial no longer needed, principal investigator left, no reason given, or other). We considered trials terminated for safety or efficacy to be completed trials. Trials marked as terminated for other reasons were considered failed trials. We then estimated the rate of trial failure using competing risks methods. Finally, we assessed associations with trial failure using a Cox proportional hazards model.
RESULTS:
A total of 1,869 urologic oncology trials were included. Of these, 225 (12.0%) failed, and 51 (2.7%) were terminated for ‘good’ reasons (e.g. toxicity, efficacy). Of the 225 failed trials, 122 (54%) failed due to poor accrual. Failed trials had a lower anticipated accrual than successfully completed trials (55 vs 63 patients, p <0.001). A total of 6,832 patients were actually accrued to failed trials. The 10-year estimated risk of trial failure was 17% [95% CI 15–22%]. Single center trials, phase 3 trials, drug trials, and trials with exclusively USA sites were more likely to fail.
CONCLUSION:
We estimate that 17%, or roughly 1 in 6, of urologic oncology trials fail, most frequently for poor accrual. Further investigations are needed into systemic, trial, and site-specific factors that may impact accrual and successful trial completion.
Keywords: Clinical trials, evidence generation, trial accrual
1. INTRODUCTION
Clinical trials are integral to evidence generation in order to inform clinical decisions. Even in the “big data” age, data from well-run trials are necessary to establish reliable comparative effectiveness data, as observational studies have not yet been shown to be reliable surrogates for treatment efficacy.[1] However, initiating, executing, and completing a well-run trial requires significant investments of time, money, and effort on behalf of multiple stakeholders. As a result, trials are often difficult to complete, and cancer clinical trials have previously been estimated to terminate for uninformative reasons (e.g., poor accrual) about 20% of the time.[2,3] Trials that terminate without reaching their prespecified endpoints are unlikely to answer the question for which they were designed, are considered failed, and make limited contributions to scientific knowledge despite significant effort.
To improve trials, the scope of these problems must be better understood and addressed. In general, the literature surrounding early trial termination for urologic oncology is either outdated or applies to cancer trials more broadly. A recent publication highlighted associations with successful trial completion in urology, but did not provide oncology-specific analyses, estimates of trial failure rates, or include newly available data points such as anticipated accrual for all trials.[4] Moreover, other studies tend to rely on early ClinicalTrials.gov data rather than contemporary urologic oncology therapeutics.
For these reasons, we examined contemporary urologic oncology trials found on ClinicalTrials.gov to estimate the rate of trial failure, reasons for failure, and associations with failure for urologic malignancies. We improve upon existing studies by updating both the methodology and available data for trial failure analysis. We believe these findings will be useful in improving clinical trial accrual and conduct, and in future analyses of clinical trial failure.
2. METHODS
2.1. Clinical trial search and criteria
On April 20, 2020, we queried ClinicalTrials.gov for adult genitourinary trials listed as phase 2/3. Search terms included “prostate cancer”, “kidney cancer”, “bladder cancer”, “urothelial cancer”, “testis cancer”, “penile cancer”, and “ureter cancer”. We included “urothelial cancer” to reflect both upper tract disease and the use of this term in the metastatic setting. We did not search explicitly for “urethral cancer”, but these studies are mostly categorized as “urothelial cancer” in our search strategy. We did not exclude trials evaluating multiple tumor types. We limited to trials starting in 2007 or later, corresponding to United States Food and Drug Administration (FDA) and International Committee of Medical Journal Editors (ICMJE) trial registration requirements.[5] We excluded trials with start dates after 4/20/2019 to allow for at least one year of enrollment prior to analysis. We downloaded and extracted trial data in XML format using a Python algorithm (Supplemental data). Anticipated and actual accrual were extracted from primary records or the ClinicalTrials.gov site via a validated and published algorithm.[6]
We excluded trials with “not yet recruiting”, “suspended”, “withdrawn”, or “unknown” status, as these trials have unclear termination status. We also excluded trials without recorded anticipated accrual or without a recorded date of completion/termination. This resulted in a total of 1,869 included studies.
2.2. Trial classification and termination outcomes
For terminated trials, three coders (KDS, KS, JR) classified termination reason into the following eleven categories: poor accrual, interim results, toxicity/adverse events, study agent unavailable, canceled by the sponsor, inadequate budget, logistics, trial no longer needed, principal investigator left, no reason given, or other. To better understand implications for future trial planning and accrual, we categorized termination into “good” and “bad” reasons. We considered termination for interim results or toxicity/adverse events to be a “good” reason for early trial closure, as these reasons are informative and should in theory answer the study question. We also considered trial termination because it was “no longer needed” as a “good” reason for early trial closure as a futile trial is no longer ethical and should be ended. We report descriptive statistics for trials terminated for “good” reasons separately as this is a distinct trial outcome, but for the primary outcome of trial failure we considered trials terminated for “good” reasons as “completed” for the purposes of censoring and competing risks analysis. In contrast, we defined trial failure as trials terminated early for other reasons as these trials are likely non-informative.
We classified a trial as “multicenter” if it listed more than one address within its clinical site record. We classified a trial as international only if it did not contain a United States address in its site list, as international and USA if it included sites within and outside the USA, and as USA only if only USA sites were included. Trials listed as “Phase 1/2” were considered Phase 2, and trials listed as Phase 2/3 were considered Phase 3. We extracted all listed intervention types and allowed a trial to have multiple intervention types.
2.3. Statistical analyses
We compared characteristics of ongoing studies, completed studies, and terminated studies. Median anticipated accrual was compared across studies using the Kruskal-Wallis test and categorical variables (cancer type, start year, phase, intervention types, multicenter vs. single-center, international vs. USA only vs. USA and international, sponsors) were compared using the chi-squared test. We estimated the total number of patients accrued to trials by summing the “actual accrual” recorded for each trial.
To estimate the likelihood of failure, we calculated cumulative incidence using competing risks via the cmprsk package in R.[7] We considered survival time to be the time from the recorded trial start date to the recorded trial completion or termination date. We considered ongoing trials to be censored at the time of data download (4/20/2020), and trial completion or termination for a “good” reason as competing risks. For the estimate of the likelihood of premature termination, we considered the best risk estimate to be when the estimate remained stable (previously described in trial termination studies as the inflection point of the cumulative incidence curve).[2]
We assessed associations with trial failure via a Cox proportional hazards model, with cancer type, trial start year, phase, multicenter vs. single-center, intervention type, trial countries (USA vs. international vs. both), sponsor (industry, NIH, other US federal, or other), and anticipated accrual as covariates. Notably, we estimated cause-specific hazards in this model by censoring trials that were completed or terminated for “good” reasons.
Data wrangling was performed in Python and statistical analysis was performed in R.[8] This study was deemed exempt from our institutional IRB.
3. RESULTS
A total of 1,869 trials were included. As illustrated in Figure 1, 892 (47.7%) were active, 701 (37.5%) were completed, 51 (2.7%) were terminated for “good” reasons, and 225 (12.0%) trials failed. Of the 225 failed trials, 122 (54%) failed due to poor accrual, 19 (8%) were canceled by the sponsor, and 29 (13%) were canceled for logistical reasons (Figure 2).
Figure 1:
Trial Selection
Figure 2:
Reasons for Trial Termination
Trial characteristics are shown in Table 1. Notably, a higher proportion of failed trials were in the USA only. Trials terminated for “good” reasons were nearly all drug trials (96%) and had a higher proportion of multicenter (59%) and industry sponsorship (69%). Failed trials had a lower anticipated accrual (median 55 patients, p<0.001) than completed trials (63 patients), ongoing trials (90 patients), or trials terminated for a “good” reason (60 patients).
Table 1:
Characteristics of Included Trials
| Active 892 | Failed 225 | Completed 701 | “Good” Terminated 51 | p | |
|---|---|---|---|---|---|
| Cancer Type (%) | 0.0 02 | ||||
| Bladder | 159 (55.6] | 40 (14.0] | 82 (28.7] | 5 (1.7) | |
| Kidney | 197 (47.2] | 54 (12.9] | 150 (36.0] | 16 (3.8) | |
| Penile | 13 (72.2] | 2 (11.1] | 3 (16.7] | 0 (0.0) | |
| Prostate | 478 (44.8] | 123 (11.5] | 439 (41.1] | 28 (2.6) | |
| Testicular | 13 (37.1] | 4 (11.4] | 1 7 (48.6] | 1 (2.9) | |
| Ureter | 1 (33.3] | 1 (33.3] | 1 (33.3] | 0 (0.0) | |
| Urothelial | 31 (73.8] | 1 (2.4] | 9 (21.4] | 1 (2.4) | |
| Start Year (%) | <0.001 | ||||
| 2007 | 7 (4.7] | 25 (16.8] | 112 (75.2] | 5 (3.4) | |
| 2008 | 12 (9. 4] | 24 (18.9] | 84 (66.1] | 7 (5.5) | |
| 2009 | 12 (9.5] | 24 (19.0] | 84 (66.7] | 6 (4.8) | |
| 2010 | 14 (11.8] | 28 (23.5] | 75 (63.0] | 2 (1.7) | |
| 2011 | 21 (17.1] | 29 (23.6] | 64 (52.0] | 9 (7.3) | |
| 2012 | 33 (23.6] | 18 (12.9] | 82 (58.6] | 7 (5.0) | |
| 2013 | 38 (32.5] | 19 (16.2] | 54 (46.2] | 6 (5.1) | |
| 2014 | 69 (53.5] | 13 (10.1] | 45 (34.9] | 2 (1.6) | |
| 2015 | 71 (51.8] | 24 (17.5] | 41 (29.9] | 1 (0.7) | |
| 2016 | 122 (73.9] | 12 (7.3] | 28 (17.0] | 3 (1.8) | |
| 2017 | 194 (87.8] | 5 (2.3] | 20 (9.0] | 2 (0.9) | |
| 2018 | 229 (94.2] | 3 (1.2] | 11 (4.5] | 0 (0.0) | |
| 2019 | 70 (95.9] | 1 (1.4] | 1 (1.4] | 1 (1.4) | |
| Phase | |||||
| Phase 2 (%) | 704 (46.7) | 194 (12.9) | 567 (37.6) | 44 (2.9) | 0.064 |
| Phase 3 (%) | 188 (52.2) | 31 (8.6) | 134 (37.2) | 7 (1.9) | |
| Treatment (%) | <0.001 | ||||
| Drug/Supplement | 530 (42.4) | 160 (12.8) | 522 (41.8) | 37 (3.0) | |
| Device/Procedure | 64 (48.9) | 14 (10.7) | 50 (38.2) | 3 (2.3) | |
| Radiation | 136 (74.7) | 13 (7.1) | 31 (17.0) | 2 (1.1) | |
| Genetic/Biologic | 134 (54.7) | 31 (12.7) | 71 (29.0) | 9 (3.7) | |
| Behavioral | 11 (33.3) | 3 (9.1) | 19 (57.6) | 0 (0.0) | |
| Trial Countries (%) | <0.001 | ||||
| Non USA | 293 (48.1) | 51 (8.4) | 250 (41.1) | 15 (2.5) | |
| USA Only | 427 (46.4) | 156 (17.0) | 310 (33.7) | 27 (2.9) | |
| Both | 172 (50.6) | .8 (5.3) | 141 (41.5) | 9 (2.6) | |
| Sponsor | |||||
| Other | 617 (52.7) | 149 (12.7) | 373 (31.9) | 31 (2.6) | <0.001 |
| Industry (%) | 441 (44.5) | 123 (12.4) | 392 (39.6) | 35 (3.5) | 0.007 |
| NIH (%) | 163 (50.2) | 43 (13.2) | 112 (34.5) | 7 (2.2) | 0.494 |
| Other US Government (%) | 10 (55.6) | 2 (11.1) | 6 (33.3) | 0 (0.0) | 0.843 |
| Anticipated A ccrial (median (IQR)) | 90 (47–240) | 55 (34–108) | 63 (40–135) | 60 (40–163) | <0.001 |
| Trial Duration (median months (IQR)) | 41 (26–66) | 33 (21–51) | 50 (33–72) | 43 (27–60) | <0.001 |
We identified a total of 6,832 patients accrued to failed trials (4 failed trials did not report actual accrual). A total of 6,554 patients were accrued to trials that prematurely terminated for “good” reasons, with 2,577 patients in this group coming from a single large trial that ended for interim results. A total of 95,099 patients enrolled in completed trials (28 completed trials did not report actual accrual).
On competing risks analysis, the 10-year estimated risk of trial failure was 17% (95% CI 15–22%). The cumulative incidence of trial failure is illustrated in Figure 3. The rate of 17% was relatively stable from 9 years through 12 years (the maximum observation time in the present study). On multivariable regression, single-center trials were more likely to fail than multicenter trials (HR 1.34, 95% CI 1.02–1.77, p=0.04). Phase 3 trials (HR 1.64, 95% CI 1.01–2.66, p=0.04) and drug trials (HR 1.72, 95% CI 1.04–2.82, p=0.03) were more likely to fail than phase 2 trials and non-drug trials. Compared to trials with USA only sites, trials were less likely to fail if they had USA and non-USA sites (HR 0.34, 95% CI 0.19–0.59, p < 0.001) or exclusively non-USA sites (HR 0.54, 95% CI 0.38–0.77, p < 0.001). Increasing anticipated accrual goals were associated with lower likelihood of trial failure (HR 0.998 for each additional patient, 95% CI 0.997–0.999, p=0.002).
Figure 3:
Cumulative Incidence of Trial Failure
4. DISCUSSION
Clinical trials are currently integral to the generation of scientific knowledge. Clinical trials often fail, however, and prior reports have suggested trial failure rates of around 20%, mostly due to poor accrual.[2,3] To improve trials, understanding failure rates, reasons, and associations with failure is important to establish a baseline for failure and preliminarily identify areas within trial infrastructure for improvement.
We estimate the rate of failure of urologic oncology trials to be 17%, meaning an estimated 1 in 6 urologic oncology trials will fail, mostly due to poor accrual. This is consistent with prior studies of oncology trials in general, which estimate a 20% failure rate.[2,3] Similar to prior studies, we identify single-center trials and trials performed exclusively in the United States as more likely to fail compared to multicenter and/or international trials.
Notably, our trial failure estimate is slightly lower than prior estimates. We used a more conservative model for trial failure wherein we considered trial completion or termination for a “good” reason as competing risks as opposed to censored observations. This model decreases the estimated risk of failure; if modeled as a simple cumulative incidence function, the estimated risk of failure is closer to prior estimates of 20–25% (Supplemental Data).
We also analyzed anticipated accrual which has not been reliably available in prior studies. Studies with a larger accrual goal were less likely to fail than smaller studies. It is possible that studies with larger accrual goals have more resources available, may be of greater importance to investigators or sponsors, or perhaps have expanded inclusion criteria making accrual easier. Further studies will more closely investigate this finding.
An underlying assumption in the analysis of failed or “unsuccessful” trials is that they do not contribute significantly to the scientific knowledge base. The ultimate fate and productivity of terminated trials should be further explored. It is possible that a trial can miss its accrual goal and still answer its intended question, particularly for those reaching 80–85% of anticipated accrual.[9][10] However, if trials consistently underaccrue and still generate sufficient evidence to change practice, we must change the way we plan and implement trials to more efficiently utilize resources, minimize harm to patients, and ensure efforts are not wasted. Furthermore, if we can answer trial questions with fewer patients, then we lose equipoise at an earlier time than considered in trial planning, and we must refine our approach to trial design and planning.
It should be noted that the careful design of clinical trials also allows for meaningful evidence generation prior to the completion of a trial. The STAMPEDE trial in urologic oncology is a banner example of trial design (an adaptive trial) that has allowed for not only improvements in accrual, but for providing answers to questions prior to the anticipated completion date of the trial (2024).[11] While trial productivity (e.g., publications or clinical impact) is beyond the scope of our present study, it should be noted that even if an adaptive trial does not reach its ultimate anticipate endpoint, it has already proven valuable by providing actionable insights and adding significant scientific knowledge. Modifying trial design to provide data while the trial is ongoing adds value to the trial, and mitigates many of the negative issues associated with trial failure.
Our results highlight the need for improving accrual within urologic oncology trials. Our results suggest that trials with greater resources (multinational, multicenter, larger accrual goals) fail less frequently than other trials, but simply devoting more resources to existing trials may not be an efficient means to improving trial conduct. Increasing site funding alone, as studied in one randomized trial, may not impact accrual.[12] Interventions to improve trial accrual, such as department-specific protocols to optimize infrastructure, qualitative interventions, task forces, corrective action plans, and business models for trial conduct may be applied in other centers to improve accrual.[13][14][15] On a wider scope, initiatives such as the Accrual to Clinical Trials Network may aid in accessing more patients across multiple sites to improve accrual in the future.[16] Using new methods of consenting and accruing patients at sites that are more convenient to patients, as well as engaging community settings in clinical trials, may aid accrual, as travel burden and access to trials have been significant accrual barriers. [17][18] Additionally, telehealth may be used to accrue to, consent for, or even run trials, which has proven feasible previously.[19–21] However, system-wide or systematic methods to improve accrual have thus far not been proven reliable.[13] In other words, the implementation of clinical trials needs to be better understood and effective trial implementation strategies developed to prevent wasted time and resources, not to mention potential patient harm.
On the other end of accrual planning, accrual prediction and determining trial futility is also an important measure. An estimated 6,832 patients were enrolled in trials that failed. While these patients may have had treatment benefit, part of the ethical agreement to enrolling on a trial is advancing science, and if a trial fails then this benefit is gone. If we are able to predict that a trial is unlikely to complete, that trial should be terminated as quickly as possible so that patients interested in clinical trials can be diverted to trials that are more likely to complete.
There are limitations to our study. Trial registration on ClinicalTrials.gov is mandated by the FDA and a prerequisite for publication in an ICMJE journal, but reporting of specific variables is not specifically required and is inconsistent.[5] Our analysis relies on factors being reported accurately on ClinicalTrials.gov; these measures are subject to reporting bias leading to unmeasured selection bias in our study. Trials with unknown status (n=176) were excluded, which may impact our results. Inclusion of the few suspended trials (n=14) is unlikely to change results, and withdrawn trials (n=86) do not enroll patients so are a distinct entity from initiated and failed trials. International trials may be less likely to report, particularly for trials more likely to fail, as they are not necessarily subject to the USFDA. Trials which accurately report may represent a distinct group with different characteristics than other trials, perhaps indicating better organization and logistical support with resulting improved accrual and success rates.
Despite these limitations, we present a conservative modern estimate that reflects a high rate of urologic cancer trial failure. We hope to expand on these results to improve clinical trial accrual and conduct.
5. CONCLUSION
We estimate roughly 1 in 6 urologic oncology trials fail, most frequently for poor accrual. Further investigations are needed into implementation, trial, and site-specific factors impacting accrual and successful trial completion.
Supplementary Material
Table 2:
Cox Model for Trial Failure
| HR | Low CI | High CI | P | |
|---|---|---|---|---|
| Year | ||||
| 2007 | Reference | |||
| 2008 | 1.07 | 0.61 | 1.89 | 0.81 |
| 2009 | 1.16 | 0.66 | 2.04 | 0.62 |
| 2010 | 1.38 | 0.80 | 2.38 | 0.25 |
| 2011 | 1.38 | 0.80 | 2.38 | 0.24 |
| 2012 | 0.74 | 0.40 | 1.38 | 0.35 |
| 2013 | 0.92 | 0.50 | 1.68 | 0.77 |
| 2014 | 0.64 | 0.33 | 1.27 | 0.21 |
| 2015 | 1.36 | 0.77 | 2.43 | 0.29 |
| 2016 | 0.59 | 0.29 | 1.19 | 0.14 |
| 2017 | 0.25 | 0.09 | 0.65 | 0.01 |
| 2018 | 0.25 | 0.08 | 0.86 | 0.03 |
| 2019 | 071 | 0.09 | 5.37 | 0.74 |
| Phase | ||||
| Phase 2 | Reference | |||
| Phase 3 | 1.64 | 1.01 | 2.66 | 0.046 |
| Number of Sites | ||||
| Multicenter | Reference | |||
| Single Center | 1.34 | 1.02 | 1.77 | 0.04 |
| Intervention Type | ||||
| Biologic | 1.27 | 0.79 | 2.03 | 0.32 |
| Drug | 1.72 | 1.04 | 2.82 | 0.03 |
| Other | 1.26 | 0.82 | 1.95 | 0.3 |
| Device | 1.33 | 0.46 | 3.86 | 0.6 |
| Radiation | 0.64 | 0.35 | 1.18 | 0.15 |
| Procedure | 0.87 | 0.53 | 1.44 | 0.59 |
| Genetic | 0.76 | 0.18 | 3.17 | 0.71 |
| Supplement | 0.63 | 0.15 | 2.63 | 0.53 |
| Behavioral | 0.93 | 0.28 | 3.05 | 0.9 |
| Site Location | ||||
| USA Only | Reference | |||
| Non-USA | 0.54 | 0.38 | 0.77 | <0.001 |
| Both USA + | ||||
| International | 0.34 | 0.19 | 0.59 | <0.001 |
| Sponsor | ||||
| Other | 0.79 | 0.56 | 1.11 | 0.17 |
| Industry | 1.19 | 0.84 | 1.68 | 0.34 |
| NIH | 0.66 | 0.41 | 1.04 | 0.07 |
| Other US Govt. | 0.54 | 0.13 | 2.24 | 0.39 |
| Anticipated Accrual | 0.998 | 0.997 | 0.999 | 0.002 |
| Cancer Type | ||||
| Bladder | Reference | |||
| Kidney | 0.75 | 0.49 | 1.14 | 0.18 |
| Penile | 1.55 | 0.36 | 6.73 | 0.56 |
| Prostate | 0.76 | 0.53 | 1.1 | 0.15 |
| Testicular | 0.55 | 0.19 | 1.58 | 0.27 |
| Ureter | 1.27 | 0.16 | 9.84 | 0.82 |
| Urothelial | 0.3 | 0.04 | 2.22 | 0.24 |
Highliggts.
An estimated 1 in 6 urologic oncology trials will fail
Most urologic oncology trial failures are due to poor accrual
International and multicenter urologic oncology trials fail less frequently than USA-only or single center trials
Nearly 7,000 patients enrolled on failed urologic oncology trials
Acknowledgments
Funding support:
Dr. Stensland is supported by the National Cancer Institute T32CA180984. The views expressed in this article do not reflect the views of the federal government. Dr. Stensland was also supported for this work in part by a Lahey Clinic Robert E. Wise Foundation grant.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- [1].Soni PD, Hartman HE, Dess RT, Abugharib A, Allen SG, Feng FY, et al. Comparison of Population-Based Observational Studies With Randomized Trials in Oncology. J Clin Oncol 2019;37:1209–16. 10.1200/JCO.18.01074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Stensland KD, McBride RB, Latif A, Wisnivesky J, Hendricks R, Roper N, et al. Adult cancer clinical trials that fail to complete: an epidemic? J Natl Cancer Inst 2014;106 10.1093/jnci/dju229. [DOI] [PubMed] [Google Scholar]
- [3].Khunger M, Rakshit S, Hernandez AV, Pasupuleti V, Glass K, Galsky MD, et al. Premature Clinical Trial Discontinuation in the Era of Immune Checkpoint Inhibitors. Oncologist 2018;23:1494–9. 10.1634/theoncologist.2018-0003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Bandari J, Theisen KM, Maganty A, Davies BJ, Yabes JG, Jacobs BL. Clinical Trials in Urology: Predictors of Successes and Failures. J Urol 2020:101097JU0000000000001072. 10.1097/JU.0000000000001072. [DOI] [PubMed] [Google Scholar]
- [5].Tse T, Fain KM, Zarin DA. How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider. BMJ 2018:k1452. 10.1136/bmj.k1452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Stensland K, Kaffenberger S, Canes D, Galsky M, Skolarus T, Moinzadeh A. Assessing Genitourinary Cancer Clinical Trial Accrual Sufficiency Using Archived Trial Data. JCO Clin Cancer Inform 2020;4:614–22. 10.1200/CCI.20.00031. [DOI] [PubMed] [Google Scholar]
- [7].Gray B cmprsk: Subdistribution Analysis of Competing Risks. 2014. [Google Scholar]
- [8].R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2017. [Google Scholar]
- [9].Schroen AT, Petroni GR, Wang H, Thielen MJ, Gray R, Benedetti J, et al. Achieving sufficient accrual to address the primary endpoint in phase III clinical trials from U.S. Cooperative Oncology Groups. Clin Cancer Res 2012;18:256–62. 10.1158/1078-0432.CCR-11-1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Poola AS, Oyetunji TA, Holcomb GW, St Peter SD. Maturation of effect size during enrollment of prospective randomized trials. J Surg Res 2018;223:34–8. 10.1016/j.jss.2017.06.082. [DOI] [PubMed] [Google Scholar]
- [11].James ND, Sydes MR, Clarke NW, Mason MD, Dearnaley DP, Anderson J, et al. Systemic therapy for advancing or metastatic prostate cancer (STAMPEDE): a multi-arm, multistage randomized controlled trial. BJU Int 2009;103:464–9. 10.1111/j.1464410X.2008.08034.x. [DOI] [PubMed] [Google Scholar]
- [12].Parker C, Snyder R, Jefford M, Dilts D, Wolfe R, Millar J. A Randomized Controlled Trial of an Additional Funding Intervention to Improve Clinical Trial Enrollment. J Natl Compr Canc Netw 2017;15:1104–10. 10.6004/jnccn.2017.0150. [DOI] [PubMed] [Google Scholar]
- [13].Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev 2018;2:MR000013. 10.1002/14651858.MR000013.pub6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Porter M, Ramaswamy B, Beisler K, Neki P, Single N, Thomas J, et al. A Comprehensive Program for the Enhancement of Accrual to Clinical Trials. Ann Surg Oncol 2016;23:2146–52. 10.1245/s10434-016-5091-9. [DOI] [PubMed] [Google Scholar]
- [15].Donovan JL, Rooshenas L, Jepson M, Elliott D, Wade J, Avery K, et al. Optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the Quintet Recruitment Intervention (QRI). Trials 2016;17:283 10.1186/s13063-016-1391-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].Visweswaran S, Becich MJ, D’Itri VS, Sendro ER, MacFadden D, Anderson NR, et al. Accrual to Clinical Trials (ACT): A Clinical and Translational Science Award Consortium Network. JAMIA Open 2018;1:147–52. 10.1093/jamiaopen/ooy033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Borno HT, Zhang L, Siegel A, Chang E, Ryan CJ. At What Cost to Clinical Trial Enrollment? A Retrospective Study of Patient Travel Burden in Cancer Clinical Trials. Oncologist 2018;23:1242–9. 10.1634/theoncologist.2017-0628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Ellis S, Geana M, Griebling T, McWilliams C, Gills J, Stratton K, et al. Development, acceptability, appropriateness and appeal of a cancer clinical trials implementation intervention for rural- and minority-serving urology practices. Trials 2019;20:578 10.1186/s13063-019-3658-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Baca-Motes K, Edwards AM, Waalen J, Edmonds S, Mehta RR, Ariniello L, et al. Digital recruitment and enrollment in a remote nationwide trial of screening for undiagnosed atrial fibrillation: Lessons from the randomized, controlled mSToPS trial. Contemp Clin Trials Commun 2019;14:100318. 10.1016/j.conctc.2019.100318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].Alfredo Caceres J, Greer DM, Goldstein JN, Viswanathan A, Suarez JI, Brau L, et al. Enrollment of research subjects through telemedicine networks in a multicenter acute intracerebral hemorrhage clinical trial: design and methods. J Vasc Interv Neurol 2014;7:34–40. [PMC free article] [PubMed] [Google Scholar]
- [21].Bobb MR, Van Heukelom PG, Faine BA, Ahmed A, Messerly JT, Bell G, et al. Telemedicine Provides Noninferior Research Informed Consent for Remote Study Enrollment: A Randomized Controlled Trial. Acad Emerg Med 2016;23:759–65. 10.1111/acem.12966. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.