Abstract
Purpose:
Clinical trials require significant resources, but benefits are only realized after trial completion and dissemination of results. We comprehensively assessed early discontinuation, registry results reporting, and publication by trial sponsor and subspecialty in urology trials.
Materials and Methods:
We assessed trial registrations from 2007–2019 on ClinicalTrials.gov and publication data from PubMed®/MEDLINE®. Associations between sponsor or subspecialty with early discontinuation were assessed using Cox proportional hazards and results reporting or publication with logistic regression at 3 years after completion.
Results:
Of 8,636 trials 3,541 (41.0%) were completed and 999 (11.6%) were discontinued. Of completed trials 26.9% reported results and 21.6% were published. Sponsors included academic institutions (53.1%), industry (37.1%) and the U.S. government (9.8%). Academic-sponsored (adjusted HR 0.81, 95% CI 0.69–0.96, p = 0.012) and government-sponsored trials (adjusted HR 0.62, 95% CI 0.49–0.78, p <0.001) were less likely than industry to discontinue early. Government-sponsored trials were more likely to report (adjusted OR 1.72, 95% CI 1.17–2.54, p = 0.006) and publish (adjusted OR 1.89, 95% CI 1.23–2.89, p = 0.004). Academic-sponsored trials were less likely to report (adjusted OR 0.65, CI:0.48–0.88, p = 0.006) but more likely to publish (adjusted OR 1.72, 95% CI 1.25–2.37, p <0.001). These outcomes were similar across subspecialties. However, endourology was more likely to discontinue early (adjusted HR 2.00, 95% CI 1.53–2.95, p <0.001), general urology was more likely to report results (adjusted OR 1.54, 95% CI 1.13–2.11, p = 0.006) and andrology was less likely to publish (adjusted OR 0.53, 95% CI 0.35–0.81, p = 0.003).
Conclusions:
Sponsor type is significantly associated with trial completion and dissemination. Government-sponsored trials had the best performance, while industry and academic-sponsored trials lagged in completion and results reporting, respectively. Subspecialty played a lesser role. Lack of dissemination remains a problem for urology trials.
Keywords: clinical trials as topic, randomized controlled trials as topic, research support as topic
Clinical trials represent a key mechanism to test medical interventions and further the evidence base for patient care.1 Although clinical trials require significant financial and professional resources, the potential benefits of their findings merit the cost. Trials that discontinue early or fail to share findings following completion—either through publication or online registries—fail to serve the scientific community. Understanding which trials complete and disseminate results will inform efforts to optimize chances of success. Clinicaltrials.gov represents one of the largest international repositories of clinical trial data. The database contains nearly 50% of all registered global clinical trials and the majority of registered U.S. trials.2 Since 2007, U.S. federal law has mandated that most phase 2–4 interventional studies register in ClinicalTrials.gov.3
In urology, few studies have investigated the clinical trial landscape to characterize trial characteristics, discontinuation or dissemination. Previous studies limited their investigations to specific diseases4–7 within urology or a smaller sample of trials.8 None of the previous studies have characterized results reporting or publication among all urology clinical trials. Given the importance of clinical trials for advancing evidence-based practice, we aimed to characterize and analyze key drivers of completion, results reporting and publication among all urology trials registered to ClinicalTrials. gov between 2007 and 2019. Our large-scale analysis allowed us to assess the role of sponsorship, capture time trends and compare trial subspecialties and features.
MATERIALS AND METHODS
Data Sources and Study Cohort
We identified all trials registered in the Aggregate Analysis of ClinicalTrials.gov database9,10 as of October 29, 2019. We limited to interventional studies registered after October 1, 2007 following enactment of the U.S. Food and Drug Administration Amendments Act, which legally mandated registration of most phase 2–4 interventional studies.1,3,11,12 We identified trials relevant to urology using MeSH® terms in accordance with published protocols.9,13,14 We manually reviewed the titles, abstracts and key terms of all trials to verify inclusion and categorization into the appropriate subspecialty: andrology, endourology, female, general, oncology, pediatric or renal transplant (supplementary material, https://www.jurology.com). We obtained trial publications by linking PubMed’s MEDLINE bibliographic database (download May 23, 2020) with ClinicalTrials.gov unique identifier (NCT number) following published methods.15 The institutional review board exempted this study from oversight as all involved databases are publicly available.
Trial Features
We extracted study status (completed, ongoing, stopped early) and the following characteristics from the aggregate analysis of ClinicalTrials.gov database considered a priori to be important trial features: primary purpose (treatment, diagnostic, preventive, other), intervention type (drug/pharmaceutical, device, procedure, behavioral, other), trial phase, number arms, blinding, randomization, use of data monitoring committee, trial enrollment, region (North America, Europe, other, multiregion), number centers, submission year and sponsor.13,14,16 Employing an approach using National Library of Medicine labels similar to prior studies,13,14,16 we assessed trial sponsors and collaborators to designate industry-sponsored, U.S. government-sponsored (National Institutes of Health and other governmental organizations) or academic-sponsored trials (using host institution funds). Remaining trials were categorized as academic-sponsored trials after a random sample analysis of 2,500 remaining sponsors listed in ClinicalTrials.gov identified that 90.1% (99% CI 88.56%–91.64%) were academic institutions as defined by U.S. legal code as well as individuals or nonprofit foundations affiliated with academic institutions.17
Outcomes
We assessed all trials with a duration of more than 0 days for early discontinuation defined as “terminated,” “withdrawn,” or “suspended.”18 We then manually reviewed and assigned listed explanations to accrual, budget/staff shortage, sponsor/business decision, poor results, principal investigator departure, safety/toxicity, loss of trial relevance, logistical issues/unavailable intervention, protocol change or intention for a new future study, early success/completed objective, other reason or not supplied. “Other” included FDA clinical review, institutional review board expiration and unspecified problems with contracts, reviews, priorities or planning. For completed trials, we assessed both results reporting and earliest publication date within 3 years, corresponding to certified delayed submission under the FDA Amendments Act.1
Statistical Analysis
We obtained 2-sided Pearson chi-square tests for descriptive statistics, Fisher exact test for discontinuation reason, and compound annual growth rates with Mann-Kendall trend test for time analysis over complete years (2008–2018).
We performed unadjusted Kaplan-Meier time-to-event analysis with Log-Rank test comparing primary outcomes (early discontinuation, results reporting, and publication) by sponsor and subspecialty. Multivariate regressions controlled for trial features described above and produced adjusted HRs for early discontinuation with Cox proportional hazards and adjusted ORs for both results reporting and publication within 3 years of completion with logistic remodels. We accounted for missing data with multiple imputation with chained equations pooling 30 imputed data sets by Rubin’s rules.19 We used 2-sided statistical significance α = 0.05, and performed analysis in R 3.6.0 (R Foundation for Statistical Reporting, Vienna, Austria).
RESULTS
Study Cohort
Of 320,336 registered trials, 33% (105,654) were excluded for noninterventional design (67,552) and/or predating October 2007 (44,957). Of these, 8,636 (4.0%) were relevant to urology (fig. 1) with an estimated 1.97 million participants (supplementary table 1, https://www.jurology.com).
Figure 1.
Study cohort—selection of trials.
Characteristics
Most trials were academic-sponsored (53.1%) followed by industry (37.1%) and government-sponsored (9.8%). Overall, more than half (51.9%) were in oncology and the least (3.3%) were in pediatrics. Compared to industry, government-sponsored trials included a greater proportion pursuing oncology, pediatrics, and endourology and fewer in andrology or general urology. Academic-sponsored had the smallest proportion in oncology with larger proportions in endourology and female urology. “Treatment” comprised the most frequent purpose across sponsors. Industry and government-sponsored trials more often assessed pharmaceuticals, while academic-sponsored included more often assessed device and procedure-related trials. Academic-sponsored trials had the largest proportion with multiple arms, blinding and randomization, while government-sponsored studies were more likely to be in North America and have DMC oversight and industry had more multicenter studies (table 1).
Table 1.
Characteristics of urological clinical trials by sponsor
| No. Overall (%) | No. Industry-Sponsored (%) | No. U.S. Government-Sponsored (%) | No. Academic-Sponsored (%) | |
|---|---|---|---|---|
| Total trials | 8,636 (100) | 3,202 (37.1) | 845 (9.8) | 4,589 (53.1) |
| Subspecialty: | ||||
| Oncology | 4,478 (51.9) | 1,751 (54.7) | 648 (76.7) | 2,079 (45.3) |
| Andrology | 702 (8.1) | 329 (10.3) | 19 (2.2) | 354 (7.7) |
| Endourology | 322 (3.7) | 44 (1.4) | 18 (2.1) | 260 (5.7) |
| Female/urogynecology | 928 (10.7) | 212 (6.6) | 30 (3.6) | 686 (14.9) |
| General urology | 1,430 (16.6) | 623 (19.5) | 70 (8.3) | 737 (16.1) |
| Pediatrics | 289 (3.3) | 57 (1.8) | 39 (4.6) | 193 (4.2) |
| Renal transplant | 487 (5.6) | 186 (5.8) | 21 (2.5) | 280 (6.1) |
| Primary purpose: | ||||
| Treatment | 6,247 (72.3) | 2,724 (85.1) | 544 (64.4) | 2,979 (64.9) |
| Diagnostic | 689 (8.0) | 121 (3.8) | 101 (12.0) | 467 (10.2) |
| Prevention | 541 (6.3) | 100 (3.1) | 51 (6.0) | 390 (8.5) |
| Other* | 1,021 (11.8) | 205 (6.4) | 144 (17.0) | 672 (14.6) |
| Missing | 138 (1.6) | 52 (1.6) | 5 (0.6) | 81 (1.8) |
| Intervention type: | ||||
| Pharmaceutical | 5,388 (62.4) | 2,705 (84.5) | 594 (70.3) | 2,089 (45.5) |
| Device | 995 (11.5) | 344 (10.7) | 37 (4.4) | 614 (13.4) |
| Procedure | 831 (9.6) | 34 (1.1) | 63 (7.5) | 734 (16.0) |
| Behavioral | 400 (4.6) | 20 (0.6) | 86 (10.2) | 294 (6.4) |
| Other | 1,022 (11.8) | 99 (3.1) | 65 (7.7) | 858 (18.7) |
| Trial phase: | ||||
| 1 | 1,079 (12.5) | 586 (18.3) | 166 (19.6) | 327 (7.1) |
| 1/2–2 | 2,496 (28.9) | 1,225 (38.3) | 347 (41.1) | 924 (20.1) |
| 2/3–3 | 1,100 (12.7) | 613 (19.1) | 66 (7.8) | 421 (9.2) |
| 4 | 870 (10.1) | 348 (10.9) | 19 (2.2) | 503 (11.0) |
| Not applicable† | 3,091 (35.8) | 430 (13.4) | 247 (29.2) | 2,414 (52.6) |
| Arms: | ||||
| Single | 3,085 (35.7) | 1,213 (37.9) | 369 (43.7) | 1,503 (32.8) |
| Multiple | 5,326 (61.7) | 1,916 (59.8) | 452 (53.5) | 2,958 (64.5) |
| Missing | 225 (2.6) | 73 (2.3) | 24 (2.8) | 128 (2.8) |
| Blinding: | ||||
| Open-label | 6,055 (70.1) | 2,301 (71.9) | 674 (79.8) | 3,080 (67.1) |
| Blinded | 2,523 (29.2) | 895 (28.0) | 161 (19.1) | 1,467 (32.0) |
| Missing | 58 (0.7) | 6 (0.2) | 10 (1.2) | 42 (0.9) |
| Randomized: | ||||
| No | 3,961 (45.9) | 1,625 (50.7) | 455 (53.8) | 1,881 (41.0) |
| Yes | 4,529 (52.4) | 1,531 (47.8) | 374 (44.3) | 2,624 (57.2) |
| Missing | 146 (1.7) | 46 (1.4) | 16 (1.9) | 84 (1.8) |
| DMC oversight: | ||||
| No | 4,220 (48.9) | 1,750 (54.7) | 243 (28.8) | 2,227 (48.5) |
| Yes | 3,476 (40.3) | 1,119 (34.9) | 456 (54.0) | 1,901 (41.4) |
| Missing | 940 (10.9) | 333 (10.4) | 146 (17.3) | 461 (10.0) |
| Enrollment:‡ | ||||
| 0–9 | 770 (8.9) | 283 (8.8) | 108 (12.8) | 379 (8.3) |
| 10–49 | 2,938 (34.0) | 1,092 (34.1) | 316 (37.4) | 1,530 (33.3) |
| 50–99 | 1,779 (20.6) | 566 (17.7) | 154 (18.2) | 1,059 (23.1) |
| 100–499 | 2,522 (29.2) | 940 (29.4) | 193 (22.8) | 1,389 (30.3) |
| 500–999 | 367 (4.2) | 194 (6.1) | 41 (4.9) | 132 (2.9) |
| ≥1,000 | 236 (2.7) | 123 (3.8) | 31 (3.7) | 82 (1.8) |
| Missing | 24 (0.3) | 4 (0.1) | 2 (0.2) | 18 (0.4) |
| Region:§ | ||||
| North America | 3,997 (46.3) | 1,508 (47.1) | 768 (90.9) | 1,721 (37.5) |
| Europe | 1,933 (22.4) | 534 (16.7) | 1 (0.1) | 1,398 (30.5) |
| Other | 1,414 (16.4) | 419 (13.1) | 12 (1.4) | 983 (21.4) |
| Multiple | 555 (6.4) | 498 (15.6) | 22 (2.6) | 35 (0.8) |
| Missing | 737 (8.5) | 243 (7.6) | 42 (5.0) | 452 (9.8) |
| Facilities: | ||||
| Single | 4,908 (56.8) | 1,091 (34.1) | 487 (57.6) | 3,330 (72.6) |
| Multiple | 2,993 (34.7) | 1,869 (58.4) | 316 (37.4) | 808 (17.6) |
| Missing | 735 (8.5) | 242 (7.6) | 42 (5.0) | 451 (9.8) |
Percentages are rounded and therefore may not sum to 100.
Sponsors were determined by listed sponsor and collaborators, assessing first for industry sponsors or collaborators, then U.S. government and agencies, then academic sponsors.
All differences were statistically significant (chi-square test, p <0.001).
Other purpose included basic science, supportive care, health services research and other.
ClinicalTrials.gov states phase “not applicable” for trials without FDA defined phases, including trials of devices or behavioral interventions.
Enrollment includes actual enrollment for completed trials and anticipated enrollment for ongoing trials.
Trials taking place in a single region (North America, Europe, other) were assigned accordingly or otherwise assigned “multiple.”
Time Trends
Urology trials increased 2008–2018 at +4.1% annual growth (p = 0.003), driven by increases in academic-sponsored trials (+9.4%, p <0.001), while government and industry saw no significant change. Oncology trials increased with +4.9% annual growth, as did endourology (+14.6%), female (+7.5%), general urology (+3.8%), and pediatrics (+2.8%). Regions outside North America and Europe (“other”) had fastest growth (+7.1%), while North America (+3.1%) and Europe (+5.5%) also increased (fig. 2, supplementary table 3, https://www.jurology.com).
Figure 2.
Clinical trials in urology over time. A, number of trials overall. B, number of trials by sponsor. C, number of trials by subspecialty. D, number of trials by geographic region. E, density plot of trial enrollment. Each dot represents 1 trial. Enrollment includes actual enrollment for completed trials and anticipated enrollment for ongoing trials.
Early Discontinuation
Overall, 999 (11.6%) trials discontinued early, while 3,541 (41.0%) reached completion, with 766 (8.9%) having unknown status and the remaining 3,330 (38.6%) being ongoing (supplementary table 4, https://www.jurology.com). Kaplan-Meier analysis demonstrated significant unadjusted differences by sponsor (p = 0.007) and subspecialty (p <0.001) (fig. 3, A and B). Reason for discontinuation differed by sponsor (table 2, p <0.001) but not subspecialty (supplementary table 5, https://www.jurology.com, p = 0.2). While 15% of trials had no documented explanation, participant accrual was the most common reason overall (39%) across sponsors (industry 32%, government-sponsored 33%, academic-sponsored 46%). Industry next cited sponsor/business decision (18%), while academic-sponsored and government-sponsored trials reported budget/staff shortages (11% and 10%, respectively). All sponsors reported some discontinuation due to poor results (industry 9%, government-sponsored 8%, academic-sponsored 7%; table 2).
Figure 3.
Cumulative incidence of early discontinuation, results reporting and publication. Sponsors were determined by listed sponsor and collaborators, assessing first for industry sponsors or collaborators, then U.S. government and agencies, then academic sponsors. A, incidence of early discontinuation by sponsor type (log-rank test, p = 0.007). B, incidence of early discontinuation by subspecialty (log-rank test, p <0.001). C, incidence of results reporting by sponsor type (log-rank test, p <0.001). D, incidence of results reporting by subspecialty (log-rank test, p <0.001). E, incidence of publication by sponsor type (log-rank test, p = 0.056). F, incidence of publication by subspecialty (log-rank test, p <0.001).
Table 2.
Reason discontinued by sponsor
| No. Overall (%) | No. Industry-Sponsored (%) | No. U.S. Government-Sponsored (%) | No. Academic-Sponsored (%) | |
|---|---|---|---|---|
| Total trials | 999 (100) | 403 (40.3) | 127 (12.7) | 469 (46.9) |
| Reason discontinued: | ||||
| Accrual | 387 (38.7) | 130 (32.3) | 42 (33.1) | 215 (45.8) |
| Budget/staff shortage | 79 (7.9) | 13 (3.2) | 13 (10.2) | 53 (11.3) |
| Sponsor/business decision | 90 (9.0) | 72 (17.9) | 7 (5.5) | 11 (2.3) |
| Poor results | 79 (7.9) | 36 (8.9) | 10 (7.9) | 33 (7.0) |
| Principal investigator departure | 40 (4.0) | 10 (2.5) | 3 (2.4) | 27 (5.8) |
| Safety/toxicity | 30 (3.0) | 17 (4.2) | 3 (2.4) | 10 (2.1) |
| Loss of trial relevance* | 25 (2.5) | 7 (1.7) | 3 (2.4) | 15 (3.2) |
| Logistical issues/unavailable intervention | 37 (3.7) | 13 (3.2) | 8 (6.3) | 16 (3.4) |
| Protocol change/intended new future study | 30 (3.0) | 11 (2.7) | 5 (3.9) | 14 (3.0) |
| Early success/completed objective | 6 (0.6) | 3 (0.7) | 0 (0.0) | 3 (0.6) |
| Other reason† | 46 (4.6) | 22 (5.5) | 6 (4.7) | 18 (3.8) |
| Not supplied | 150 (15.0) | 69 (17.1) | 27 (21.3) | 54 (11.5) |
Percentages are rounded and therefore may not sum to 100.
Sponsors were determined by listed sponsor and collaborators, assessing first for industry sponsors or collaborators, then U.S. government and agencies, then academic sponsors.
All differences were statistically significant (Fisher exact test, p <0.001).
Loss of trial relevance included competing trial results and change in standard practice.
Other reasons included FDA clinical review, institutional review board expiration and unspecified problems with contracts, reviews, priorities or planning.
Adjustment for trial characteristics supported association with sponsorship: academic-sponsored were 19% (adjusted HR 0.81, 95% CI 0.69–0.96, p = 0.012) and government-sponsored trials 38% (adjusted HR 0.62, 95% CI 0.49–0.78, p <0.001) less likely than industry to discontinue early. Among subspecialties, endourology was twice as likely to discontinue relative to oncology (adjusted HR 2.00, 95% CI 1.35–2.95, p <0.001), while there were no significant differences among other subspecialties (table 3).
Table 3.
Association of sponsor and subspecialty with early discontinuation, results reporting and publication
| Adjusted HR | 95% CI | p Value | |
|---|---|---|---|
| Early discontinuation | |||
| Sponsor: | |||
| Industry | Reference | ||
| U.S. government | 0.62 | 0.49–0.78 | <0.001 |
| Academic | 0.81 | 0.69–0.96 | 0.012 |
| Subspecialty: | |||
| Oncology | Reference | ||
| Andrology | 0.78 | 0.56–1.07 | 0.1 |
| Endourology | 2.00 | 1.35–2.95 | <0.001 |
| Female | 1.22 | 0.92–1.62 | 0.2 |
| General | 1.24 | 0.99–1.56 | 0.061 |
| Pediatrics | 0.72 | 0.48–1.10 | 0.1 |
| Renal transplant | 0.89 | 0.65–1.21 | 0.4 |
| Results reporting | |||
| Sponsor: | |||
| Industry | Reference | ||
| U.S. government | 1.72 | 1.17–2.54 | 0.006 |
| Academic | 0.65 | 0.48–0.88 | 0.006 |
| Subspecialty: | |||
| Oncology | Reference | ||
| Andrology | 1.03 | 0.71–1.50 | 0.9 |
| Endourology | 1.57 | 0.81–3.04 | 0.2 |
| Female | 1.15 | 0.76–1.74 | 0.5 |
| General | 1.54 | 1.13–2.11 | 0.006 |
| Pediatrics | 1.37 | 0.77–2.46 | 0.3 |
| Renal transplant | 0.79 | 0.48–1.31 | 0.4 |
| Publication | |||
| Sponsor: | |||
| Industry | Reference | ||
| U.S. government | 1.89 | 1.23–2.89 | 0.0044 |
| Academic | 1.72 | 1.25–2.37 | <0.001 |
| Subspecialty: | |||
| Oncology | Reference | ||
| Andrology | 0.53 | 0.35–0.81 | 0.003 |
| Endourology | 0.51 | 0.24–1.08 | 0.078 |
| Female | 0.88 | 0.60–1.29 | 0.5 |
| General | 0.87 | 0.64–1.19 | 0.4 |
| Pediatrics | 0.96 | 0.55–1.66 | 0.9 |
| Renal transplant | 0.89 | 0.55–1.45 | 0.6 |
Adjusted HRs and adjusted ORs were obtained by multivariate Cox proportional hazard and logistic regression models, respectively. Adjusted ORs were assessed at 3 years following trial completion.
Modeled estimates are adjusted for primary purpose, intervention type, trial phase, arms, blinding, randomization, DMC, enrollment, number of facilities, region and first submission year. Covariate estimates are provided in supplementary tables 7–9, https://www.jurology.com.
Sponsors were determined by listed sponsor and collaborators, assessing first for industry sponsors or collaborators, then U.S. government and agencies, then academic sponsors.
Results Reporting
Of completed trials 953 (26.9%) reported results to the registry, with government-sponsored (45.8%) and industry (36.6%) reporting more than academic-sponsored (14.0%, supplementary table 6, https://www.jurology.com). Sponsor and subspecialty had significant (p <0.001) unadjusted differences on Kaplan-Meier analysis (fig. 3, C and D). After adjustment, government-sponsored trials were more likely than industry to report within 3 years of completion (adjusted OR 1.72, 95% CI 1.17–2.54, p = 0.006), while academic-sponsored were less likely (adjusted OR 0.65, 95% CI 0.48–0.88, p = 0.006). There was no significant association for most subspecialties compared to oncology, except general urology was more likely to report results (adjusted OR 1.54, 95% CI 1.13–2.11, p = 0.006; table 3).
Publication
Of completed trials 764 (21.6%) published, with a greater proportion seen in government-sponsored trials (26.3%) compared to industry (21.0%) and academic-sponsored (21.3%). Of all trials 1,424 (40.2%) either reported results or published, while only 293 (8.3%) did both (supplementary table 6, https://www.jurology.com). Kaplan-Meier analysis demonstrated trends that did not achieve statistical significance by sponsor (p = 0.056) but did by subspecialty (p <0.001, fig. 3, E and F). Multivariate logistic regression demonstrated association with sponsorship: both academic-sponsored (adjusted OR 1.72, 95% CI 1.25–2.37, p <0.001) and government-sponsored (adjusted OR 1.89, 95% CI 1.23–2.89, p = 0.004) trials were more likely to publish than industry within 3 years. While most specialties demonstrated no significant difference compared to oncology, andrology (adjusted OR 0.53, 95% CI 0.35–0.81, p = 0.003) remained less likely to publish by 3 years (table 3).
DISCUSSION
To our knowledge, this study is the largest and most comprehensive assessment of clinical trials within urology. We found trial sponsor is associated with study completion and dissemination of trial findings through results reporting to the ClinicalTrials.gov registry or peer-reviewed publication. Government sponsorship had the best association with trial completion and dissemination through registry reporting or publication, while industry was more likely to discontinue and academic-sponsored trials were less likely to report results to the registry. Subspecialty had fewer associations, with most performing similarly to oncology except for higher risk of discontinuation with endourology, better results reporting for general urology, and less publication in andrology. As clinical trials represent enormous investments of resources on behalf of investigators and patient participation, it is important to understand if these efforts deliver on their promised benefits through trial completion and sharing results. The identified influence of sponsorship on trial outcomes provides key insights into areas for trial improvement and optimization of trial success given that sponsor incentives and processes for choosing trials differ.
Our study makes unique contributions by assessing results reporting and publication for the first time in urology. There have been 5 prior studies assessing urology trials that were more limited in scope (with 3 studies examining fewer than 125 trials), restricted to a single disease focus or limited in date range or type of trial.4–8 The other studies’ smaller sample size may reflect the analysis challenges of ClinicalTrials. gov,9,11 with free text or redundant data fields often requiring cumbersome manual review.13,14 One recent investigation applied limited search terms to a smaller sample of only 1,340 trials and conducted a less comprehensive analysis.8 In contrast to our results, that investigation did not find a difference in trial failure between industry and government-sponsored trials, which likely is due to methodological differences. Unlike that study, we thoroughly parsed trials with multiple listed sponsors in order to, for example, capture industry or government partnerships with academic referral centers. Furthermore, our use of multivariate Cox regression captures more information through time-to-event analysis than discrete logistic regression, and we included several markers of trial quality as covariates (eg DMC, number of arms, blinding). Finally, we used rigorous published informatics protocols that avoid common pitfalls to capture more than a sixfold larger set of urology trials.9,11,13,14
Our results are consistent with work assessing trials in other medical specialties. We found urology trials had an overall unadjusted 11.6% rate of early discontinuation, and among completed trials 26.9% reported results and 21.6% published. Allowing for either results reporting or publication, overall dissemination was 40.2%, while only 8.3% both published and reported. Work in radiation oncology found a similar 10.1% discontinuation, and samples of oncology trials have found ranges from 12.7% to as high as 20%.13,20 Work in other subspecialties have also found industry-sponsored trials more likely to discontinue than those with federal funding14,20 and less likely to publish.21 A widely heterogeneous literature has examined lack of results dissemination as well as assessed discrepancies in data reported to ClinicalTrials.gov vs peer-reviewed publication.22–24 Studies specific to oncology have found publication at 17.6%,21 while dissemination was lower in obstetrics with 11% reporting results and under 7% publishing.25 Work examining all trials regardless of specialty have found varied estimates depending on the precise sample or date range used, with reporting compliance ranging from 13%–41% at 12 months up to an overall incidence of 38%–66%,1,16,22,26 while publication has been reported at 27%. Meanwhile, a sample of high volume pharmaceutical companies with successful FDA approvals found 56% published but only 20% reported results,27 while 22% of drug focused trials reported results in another study.28
While it is encouraging that early discontinuation appears to occur in only a minority of trials in urology, it is concerning that dissemination of results remains poor with only one-quarter of trials reporting results to the registry and only one-fifth publishing. Urology is not an outlier given low reporting is a problem documented in studies across the entire ClinicalTrials.gov database irrespective of medical specialty.1 Incentives under the FDA Amendments Act and other reforms intended to increase clinical trial transparency have likely been undermined by the absence of enforcement by regulators.26 It is extremely concerning that so few trials in urology go on to share results, and urgent efforts at improvement are necessary.
Academic-sponsored trials offer an opportunity for urologists to improve trial completion and reporting. Challenges known to impact academic-sponsored trials include access to only part-time staff and sometimes new, inexperienced investigators who may struggle to comply with the requirements and procedures for registry reporting, unlike industry where structured support and familiarity with the process are the norm, or government-sponsorship, which entails additional oversight.12 Academic centers could consider pooling resources to form ‘core facilities’ available to investigators hoping to optimize trials for success. These centralized resources could include training opportunities, facilitated mentorship between new and experienced investigators, statistical resources, and professional staffing familiar with registry compliance requirements ready to assist with results submission. Furthermore, just as journals have required prospective registration of trials as a condition for article submission, they can encourage or require results reporting to registries at the time of publication. We found academic-sponsored trials had particular challenges with discontinuation attributed to accrual. Resources should be devoted to understanding the underlying reasons for these failures, such as review of enrollment and exclusion criteria to see whether trials are too restrictive, and analysis of practice populations and providers to see whether eligible patients are simply not being referred to trials. Targeted interventions in the form of resources to help with recruitment or improvements in trial design could then be applied so that academic centers become beacons of trial transparency while improving the quality and reporting of ongoing and future research.
We must note several limitations with our study. First, ClinicalTrials.gov represents only a sample of all clinical trials and more reliably represents U.S. based trials than elsewhere. However, it contains the majority of U.S. trials and a large sample—nearly half—of registered global clinical trials.2 All other limitations with the registry also apply as described elsewhere, including dependence on trial self-reporting and occasional changes to database structure over time.1,2,11,13,14,16,26 Second, we conduct multiple analyses in this study and therefore findings close to the 0.05 significance threshold should be interpreted with acknowledgement that multiple testing increases the risk of type I error.
CONCLUSIONS
This study is the largest and most comprehensive characterization of urology registrations to ClinicalTrials.gov to date. We found study sponsor is a key driver in trial outcomes. Government-sponsored trials were the least likely to discontinue early, most likely to report results and published similarly to academic-sponsored trials. Industry had the highest risk of discontinuation and was least likely to publish, while academic-sponsored trials were least likely to report results. While early discontinuation was infrequent, results dissemination via publication or registry reporting was surprisingly low, which must urgently be improved.
Supplementary Material
Abbreviations and Acronyms
- DMC
data monitoring committee
- FDA
U.S. Food and Drug Administration
REFERENCES
- 1.Zarin DA, Fain KM, Dobbins HD et al. : 10-Year update on study results submitted to ClinicalTrials.gov. N Engl J Med 2019; 381: 1966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Banno M, Tsujimoto Y and Kataoka Y: Studies registered in non-ClinicalTrials.gov accounted for an increasing proportion of protocol registrations in medical research. J Clin Epidemiol 2019; 116: 106. [DOI] [PubMed] [Google Scholar]
- 3.Public Law 110–85: Food and Drug Administration Amendments Act of 2007. Available at: https://www.govinfo.gov/content/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf.
- 4.Rencsok EM, Bazzi LA, McKay RR et al. : Diversity of enrollment in prostate cancer clinical trials: current status and future directions. Cancer Epidemiol Biomarkers Prev 2020; 29: 1374. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Balakrishnan AS, Palmer NR, Fergus KB et al. : Minority recruitment trends in phase III prostate cancer clinical trials (2003 to 2014): progress and critical areas for improvement. J Urol 2019; 201: 259. [DOI] [PubMed] [Google Scholar]
- 6.Kunath F, Krause SF, Wullich B et al. : Bladder cancer—the neglected tumor: a descriptive analysis of publications referenced in MEDLINE and data from the register ClinicalTrials.gov. BMC Urol 2013; 13: 56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Tsikkinis A, Cihoric N, Giannarini G et al. : Clinical perspectives from randomized phase 3 trials on prostate cancer: an analysis of the ClinicalTrials.gov database. Eur Urol Focus 2015; 1: 173. [DOI] [PubMed] [Google Scholar]
- 8.Bandari J, Theisen KM, Maganty A et al. : Clinical trials in urology: predictors of successes and failures. J Urol 2020; 204: 805. [DOI] [PubMed] [Google Scholar]
- 9.Tasneem A, Aberle L, Ananth H et al. : The database for aggregate analysis of ClinicalTrials.gov (AACT) and subsequent regrouping by clinical specialty. PLoS One 2012; 7: e33677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Clinical Trials Transformation Initiative: Improving public access to aggregate content of ClinicalTrials.gov. Available at: https://aact.ctti-clinicaltrials.org/. Accessed October 29, 2019.
- 11.Tse T, Fain KM and Zarin DA: How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider. BMJ 2018; 361: k1452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tse T, Williams RJ and Zarin DA: Reporting “basic results” in ClinicalTrials.gov. Chest 2009; 136: 295. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Liu X, Zhang Y, Tang LL et al. : Characteristics of radiotherapy trials compared with other oncological clinical trials in the past 10 years. JAMA Oncol 2018; 4: 1073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Turner B, Rajeshuni N, Tran EM et al. : Characteristics of ophthalmology trials registered in ClinicalTrials.gov, 2007–2018. Am J Ophthalmol 2020; 211: 132. [DOI] [PubMed] [Google Scholar]
- 15.Zwierzyna M, Davies M, Hingorani AD et al. : Clinical trial design and dissemination: comprehensive analysis of ClinicalTrials.gov and PubMed data since 2005. BMJ 2018; 361: k2130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Anderson ML, Chiswell K, Peterson ED et al. : Compliance with results reporting at ClinicalTrials.gov. N Engl J Med 2015; 372: 1031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.20 U.S. Code § 1001: General definition of institution of higher education. Available at: https://www.law.cornell.edu/uscode/text/20/1001.
- 18.ClinicalTrials.gov: Glossary of common site terms. Available at: https://clinicaltrials.gov/ct2/about-studies/glossary. Accessed June 21, 2020.
- 19.van Buuren S and Groothuis-Oudshoorn K: Mice: multivariate imputation by chained equations in R. J Stat Softw 2011; 45: 1. [Google Scholar]
- 20.Stensland KD, McBride RB, Latif A et al. : Adult cancer clinical trials that fail to complete: an epidemic? J Natl Cancer Inst 2014; 106: dju229. [DOI] [PubMed] [Google Scholar]
- 21.Ramsey S and Scoggins J: Commentary: practicing on the tip of an information iceberg? Evidence of underpublication of registered clinical trials in oncology. Oncologist 2008; 13: 925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Chen R, Desai NR, Ross JS et al. : Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ 2016; 352: i637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Becker JE, Krumholz HM, Ben-Josef G et al. : Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 2014; 311: 1063. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hartung DM, Zarin DA, Guise JM et al. : Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann Intern Med 2014; 160: 477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Stockmann C, Sherwin CMT, Koren G et al. : Characteristics and publication patterns of obstetric studies registered in ClinicalTrials.gov. J Clin Pharmacol 2014; 54: 432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.DeVito NJ, Bacon S and Goldacre B: Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. Lancet 2020; 395: 361. [DOI] [PubMed] [Google Scholar]
- 27.Miller JE, Korn D and Ross JS: Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open 2015; 5: e009758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Prayle AP, Hurley MN and Smyth AR: Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012; 344: d7373. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.