Abstract
This study evaluates the time between submission of trial results to the National Institutes of Health and public posting of the results.
The US Food and Drug Administration Amendments Act of 2007 (FDAAA)1 requires that applicable clinical trials (ACTs) submit results to ClinicalTrials.gov within 1 year of completion. ClinicalTrials.gov identifies trials that likely meet this definition as probable ACTs (pACTs). A complementary National Institutes of Health (NIH) policy requires that nonapplicable clinical trials (non-ACTs) funded by grants submitted to NIH from January 18, 2017, also submit results.2
After investigators submit results, the National Library of Medicine reviews their quality and may request changes by sending “comments” to investigators. Whether the NIH comments on records, the NIH must post results within 30 calendar days of first submission; the FDAAA makes no exemption for quality review time, and the NIH has not implemented a process for posting records that do not pass review.1,3 We evaluated the time between submitting and public posting of results.
Methods
We examined all records in the “JohnsHopkinsU” ClinicalTrials.gov account, with results submitted from January 1, 2017, to December 31, 2018. JohnsHopkinsU includes studies conducted by the Johns Hopkins University (JHU) School of Medicine and School of Nursing. Other parts of JHU, including Bloomberg School of Public Health and Sidney Kimmel Comprehensive Cancer Center, have separate ClinicalTrials.gov accounts. Records are categorized as ACTs, pACTs, or non-ACTs. We combined the ACTs with the pACTs in our analysis. We determined the number of submission cycles and the number of days between submission and public posting, including the number of days under review by NIH and JHU. The first submission cycle began the day investigators submitted results and concluded when results were either posted by NIH or the record was returned to JHU with comments. Records publicly posted by the NIH to ClinicalTrials.gov by January 7, 2019, were included in the analysis. We used information available publicly and only to JHU, with data and code available at https://osf.io/9paqz/. Analyses were performed using Stata, version 15.1 (StataCorp) and were repeated stratified by record type, pACT (including ACTs) vs non-ACT, because non-ACT investigators and the NIH might give lower priority to voluntary submissions compared with required submissions. Because the analysis did not involve human subjects, there was no institutional review board submission, and patient consent was not obtained.
Results
Of 121 records submitted, we analyzed 115 records that posted results by January 7, 2019, including 97 of 115 pACTs (84%) and 18 of 115 non-ACTs (16%) (Table). Johns Hopkins University submitted records 1 to 5 times (mean [SD] = 2.25 [0.79]). The NIH sent first comments a mean (SD) of 32.68 (6.97) days after submission for pACTs and 63.19 (52.99) days after submission for non-ACTs. On average, pACTs were posted 76.23 (39.53) days after first submission; non-ACTs were posted after 162 (139.85) days. The NIH posted 7 of 97 pACTs (7%) within 30 days.
Table. Time From Submission of Johns Hopkins University Clinical Trial Results to Results Posted, 2017-2018.
| Variable | FDAAA Status, Mean (SD) | |
|---|---|---|
| ACT/pACTa | Non-ACT | |
| Records with posted results, No. | 97 | 18 |
| Submission cycles, No. | ||
| Mean (SD) | 2.26 (0.81) | 2.22 (0.73) |
| Median (IQR) | 2 (1) | 2 (1) |
| Total time in review, d | ||
| JHU | 16.84 (22.64) | 43.72 (56.97) |
| NIH | 59.39 (26.07) | 118.28 (98.52) |
| Total | 76.23 (39.53) | 162.00 (139.85) |
| No. of days to receive first commentsb | 32.68 (6.97) | 63.19 (52.99) |
| Posted after first cycle, No./total No. (%) | 12/97 (12) | 2/18 (11) |
| Time to review in first cycle, d | ||
| JHU | NA | NA |
| NIH | 30.92 (10.06) | 30.00 (1.41) |
| Not posted after first cycle, No. | 85 | 16 |
| Posted after second cycle, No./total No. (%) | 57/85 (67) | 11/16 (69) |
| Time to review in second cycle, d | ||
| JHU | 17.51 (22.58) | 62.09 (66.08) |
| NIH | 20.65 (12.76) | 77.64 (93.72) |
| Not posted after second cycle, No. | 28 | 5 |
| Posted after third cycle, No./total No. (%) | 20/28 (71) | 4/5 (80) |
| Time to review in third cycle, d | ||
| JHU | 4.20 (4.80) | 12.75 (22.19) |
| NIH | 17.90 (12.74) | 8.75 (13.10) |
| Not posted after third cycle, No. | 8 | 1 |
| Posted after fourth cycle, No./total No. (%) | 7/8 (88) | 1/1 (100) |
| Time to review in fourth cycle, d | ||
| JHU | 1 (0)c | 1 (NA)d |
| NIH | 16.14 (15.12) | 0 (NA) |
| Not posted after fourth cycle, No. | 1 | 0 |
| Posted after fifth cycle, No./total No. (%) | 1/1 (100) | 0/0e |
| Time to review in fifth cycle, d | ||
| JHU | 33 (NA) | NA |
| NIH | 32 (NA) | NA |
Abbreviations: ACT, applicable clinical trial; FDAAA, US Food and Drug Administration Amendments Act of 2007; IQR, interquartile range; JHU, Johns Hopkins University; NA, not applicable; NIH, National Institutes of Health; Non-ACTs, nonapplicable clinical trials; pACTs, probable applicable clinical trials.
The FDAAA requires results reporting for ACTs. ClinicalTrials.gov identifies trials that likely meet this definition as pACTs. We combined the ACTs with the pACTs in our analysis.
If the NIH decides not to post a record, the NIH sends comments to the investigators, which ends one cycle and begins the next. In this analysis, we counted the day on which NIH comments were received as a JHU day.
All 7 studies posted after the fourth submission cycle were modified and returned to NIH on the same day JHU received comments.
Because there is 1 record, the SD is not applicable.
All records were posted before this review cycle.
Discussion
The NIH took more than 30 days to comment on most clinical trial results submitted by JHU in 2017 and 2018. Furthermore, multiple submission cycles delayed posting of results (Figure). Although NIH quality review may ensure that clinical trial results are understandable and accurate, federal law requires that the NIH post results within 30 days without exception for quality review. For 93% of JHU records, the NIH exceeded the 30-day statutory limit for posting. The NIH might be able to post records faster as organizations that conduct clinical trials learn to identify errors and inconsistencies before submitting their results.4 Several steps could be taken to ensure both that results are accurate and that they are posted in a timely manner. First, the NIH could summarize the types of comments that lead to multiple submissions, and the NIH could publish information about the duration of quality review, as we have done in this report. Sharing this information could help organizations that conduct clinical trials improve reporting. Second, academic organizations could support investigators by updating their policies and procedures and by hiring staff to support trial registration and reporting.5 Administrators who interact with ClinicalTrials.gov regularly could support investigators who interact with ClinicalTrials.gov infrequently. Finally, the NIH could further improve automatic checking and reduce the time to review submitted records.
Figure. Days From Submission to Results Posted on ClinicalTrials.gov for Johns Hopkins University Clinical Trial Results Submitted 2017-2018.
This figure shows the average number of days that records were in review by the National Institutes of Health (NIH) in light blue and Johns Hopkins University (JHU) in dark blue for records approved after a given number of submission cycles. The number below each bar is the number of records for which results were posted by NIH at the end of a given cycle (the Table shows the proportion of records that were posted after each number of cycles). Johns Hopkins University submitted results to ClinicalTrials.gov for 97 probable applicable clinical trials (pACTs) and 24 nonapplicable clinical trials (non-ACTs) (6 of these non-ACTS had not been posted by January 7, 2019, and are not included in the analysis). Thus, the first bar shows the average number of days in review for the 12 of 97 pACTs that were posted after first submission. Of pACTs submitted, 85 were submitted 2 or more times; the third bar shows the average number of days in review for the 57 of 85 pACTs that were posted after their second submission.
References
- 1.Food and Drug Administration Amendments Act of 2007 (FDAAA), Title VIII—Clinical Trial Databases, SEC. 801. Expanded clinical trials registry data bank. https://www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf#page=82. Accessed May 8, 2018.
- 2.National Institutes of Health NIH policy on dissemination of NIH-funded clinical trial information. Fed Regist. 2016;81(183). [Google Scholar]
- 3.Department of Health and Human Services 42 CFR part 11: clinical trials registration and results information submission; final rule. https://www.gpo.gov/fdsys/pkg/FR-2016-09-21/pdf/2016-22129.pdf. Published September 21, 2016. Accessed May 8, 2018. [PubMed]
- 4.ClinicalTrials.gov results and document upload quality control review criteria. https://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf. Published June 27, 2018. Accessed April 9, 2019.
- 5.Mayo-Wilson E, Heyward J, Keyes A, et al. ; National Clinical Trials Registration and Results Reporting Taskforce Survey Subcommittee Clinical trial registration and reporting: a survey of academic organizations in the United States. https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1042-6. Accessed May 8, 2018. [DOI] [PMC free article] [PubMed]