Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
letter
. 2019 Nov 12;35(10):3105–3107. doi: 10.1007/s11606-019-05523-4

Research Waste in Randomized Clinical Trials: a Cross-Sectional Analysis

Alexander R Zheutlin 1,, Joshua Niforatos 2, Eric Stulberg 1, Jeremy Sussman 3
PMCID: PMC7572905  PMID: 31720964

INTRODUCTION

Randomized clinical trials (RCT) require substantial investments from patients, investigators, and funding agencies. RCTs that fail to be completed or to report findings represent research waste, and unnecessary patient harm. Determining RCT viability is critical in allocating funds, and ClinicalTrials.gov provides a platform for investigators to post results from completed RCTs irrespective of “success,” which can influence publication.1 We sought to describe research waste in Phase 3 RCTs as indicated by inability to reach completion and insufficient reporting of results when completed.

METHODS

We examined Phase 3 (defined by ClinicalTrials.gov), interventional (non-observational), parallel, two-arm (≥ three excluded) RCTs conducted entirely in the United States. RCTs abstracted from ClinicalTrials.gov were first registered between January 1, 2013, and December 31, 2014. Dates were chosen to allow for sufficient post-completion publication to the time of abstraction. RCTs were categorized as discontinued before completion (terminated, withdrawn, or suspended), completed without results, and completed with results. Data extracted from each RCT included number of study locations, sponsor type, therapeutic modality, RCT completion status, estimated sample size, actual enrollment, availability of results on the registry, and availability of results in a manuscript. When this data was not provided in ClinicalTrials.gov, we attempted to extract it from the study protocol or published manuscript linked to the registry when possible. We compared completion rates of RCTs using χ2 tests. Statistical analyses were performed using R 3.5.3.

RESULTS

Three hundred seventy-three RCTs in ClinicalTrials.gov met inclusion criteria (Table 1). Overall, 90 RCTs (24.1%) were discontinued before completion; 101 (27.1%) were completed without results in ClinicalTrials.gov; and 182 (48.8%) were completed with results available (Table 1). For discontinued RCTs, there was no difference in intended sample size with RCTs that were completed with results or completed without results (p = 0.37 and p = 0.49, respectively). RCTs that were discontinued had lower rates of meeting enrollment than both RCTs completed with results (p < 0.001) and completed without results (p < 0.001). Completed studies with and without results met enrollment at similar rates (p = 0.79). No discontinued RCTs were published. 108 (59.3%) of the RCTs completed with results reported in the registry were published in the peer-reviewed literature, while 22 of the 101 (21.78%) RCTs completed without results reported in the registry had results available in the published peer-reviewed literature (p < 0.001).

Table 1.

Descriptive Statistics of RCTs based upon Trial Status

Discontinued before Completion
90 (24.13%)
Completed Trials without Results
101 (27.08%)
Completed Trials with Results
182 (48.79%)
Duration (SD), years 2.06 (1.25) 2.19 (1.44) 1.62 (1.00)
Trial Completed > 12 Months Before Abstraction Date (2/15/2019) NA 86 (85.15%) 176 (96.70%)
Sponsor
  Private 61 (67.78%) 74 (73.27%) 159 (87.36%)
  Public 29 (32.22%) 27 (26.73%) 23 (12.64%)
Multi-site 56 (62.22%) 74 (73.27%) 158 (86.81%)
Median Intended Sample Size (IQR), n 284.0 (100.0–500.0) 311.0 (120.0–622.5) 304 (142.5–617.0)
Median Enrolled Sample Size (IQR), n 54.0 (6.0–373.2) 304 (95.0–535.0) 342.5 (149.2–711.0)
Met Intended Sample Size (%), n of trials 17 (18.89%) 74 (73.27%) 145 (79.67%)
Intervention Type
  Drug 83 (92.22%) 90 (89.11%) 176 (96.70%)
  Device 1 (1.11%) 2 (1.98%) 3 (1.65%)
  Surgery 1 (1.11%) 2 (1.98%) 1 (0.55%)
  Other 5 (5.56%) 7 (6.93%) 2 (1.10%)
Published Manuscript 0 22 (21.78%) 108 (59.34%)
Rationale For Status
  No Explanation 28 (31.11%) Completed Completed
  Poor Accrual 19 (21.11%) Completed Completed
  Futility 18 (20.00%) Completed Completed
  Sponsor Decision 7 (7.78%) Completed Completed
  Study Design Concerns 6 (6.67%) Completed Completed
  Clinical Hold 4 (4.44%) Completed Completed
Insufficient Funding 3 (3.33%) Completed Completed
Safety Concern 3 (3.33%) Completed Completed
  Study Drug Discontinued 2 (2.22%) Completed Completed

Of the 90 discontinued RCTs, 28 (31.1%) provided no information regarding why the RCT was discontinued. Full rationales provided can be seen in Table 1.

Discontinued RCTs were more likely to be single site (p < 0.0001) and publicly funded (p < 0.001) compared to RCTs completed with results, however, not compared to RCTs completed without results (p = 0.08 and p = 0.41, respectively) (Fig. 1).

Fig. 1.

Fig. 1

Frequency of clinical trials completion status by single- vs. multiple-sites and sponsor.

DISCUSSION

We found that ¼ of all Phase 3, parallel, two-arm RCTs registered to ClinicalTrials.gov in 2013 and 2014 were incomplete. RCTs that were discontinued before completion were disproportionately publicly funded. They were also much more likely to be single site, which public sources were more likely to fund.1, 2 Another quarter were completed but not reported on ClinicalTrials.gov, despite years after trial completion.

Our results updates prior work showing discontinued RCTs cause substantial research waste.3, 4 This study focused exclusively on Phase 3 RCTs, which require the greatest investment of human capital and are most likely to alter practice. Furthermore, our work categorizes RCTs by discontinued before completion, completed with results, and completed without results, which allows inference into uninformed waste through lack of completion and reporting of Phase 3 RCTs.

This study is limited by the use of reporting parameters, including trial phase, on ClinicalTrials.gov. ClinicalTrials.gov relies on RCTs using the unique NCT code for identification when cataloging studies that have been published. Reliance on ClinicalTrials.gov and trialist reporting may create selection bias against those who report diligently. We also lack information on the specific impact of these discontinued RCTs.

Operational difficulties with RCTs, including failure to accrue and failure to meet intended sample size, create inefficiencies and put patients to risk without benefit.5 Current movements to improve the operations of RCTs, particularly in single-site, smaller trials, could minimize this risk.2 Understanding drivers of RCTs that lead to insufficient accrual, failure to complete, or results reporting is critical to both propagating sound science, and eradicating research waste.

Funding Information

This work is supported by VA CDA 13-021 (Sussman).

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104. doi: 10.1136/bmj.f6104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.PAR-19-328: Single-Site Investigator-Initiated Clinical Trials (R61/R33 Clinical Trial Required). https://grants.nih.gov/grants/guide/pa-files/PAR-19-328.html. Accessed August 22, 2019.
  • 3.Zarin DA, Goodman SN, Kimmelman J. Harms From Uninformative Clinical Trials. JAMA. July 2019. doi:10.1001/jama.2019.9892 [DOI] [PubMed]
  • 4.Williams RJ, Tse T, DiPiazza K, et al. Terminated Trials in the ClinicalTrials.gov Results Database: Evaluation of Availability of Primary Outcome Data and Reasons for Termination. PLOS ONE. 2015;10(5):e0127242. doi:10.1371/journal.pone.0127242 [DOI] [PMC free article] [PubMed]
  • 5.Carlisle B, Kimmelman J, Ramsay T, et al. Unsuccessful Trial Accrual and Human Subjects Protections: An Empirical Analysis of Recently Closed Trials. Clin Trials. 2015;12(1):77–83. doi: 10.1177/1740774514558307. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES