Abstract
Objective To determine rates of publication and reporting of results within two years for all completed clinical trials registered in ClinicalTrials.gov across leading academic medical centers in the United States.
Design Cross sectional analysis.
Setting Academic medical centers in the United States.
Participants Academic medical centers with 40 or more completed interventional trials registered on ClinicalTrials.gov.
Methods Using the Aggregate Analysis of ClinicalTrials.gov database and manual review, we identified all interventional clinical trials registered on ClinicalTrials.gov with a primary completion date between October 2007 and September 2010 and with a lead investigator affiliated with an academic medical center.
Main outcome measures The proportion of trials that disseminated results, defined as publication or reporting of results on ClinicalTrials.gov, overall and within 24 months of study completion.
Results We identified 4347 interventional clinical trials across 51 academic medical centers. Among the trials, 1005 (23%) enrolled more than 100 patients, 1216 (28%) were double blind, and 2169 (50%) were phase II through IV. Overall, academic medical centers disseminated results for 2892 (66%) trials, with 1560 (35.9%) achieving this within 24 months of study completion. The proportion of clinical trials with results disseminated within 24 months of study completion ranged from 16.2% (6/37) to 55.3% (57/103) across academic medical centers. The proportion of clinical trials published within 24 months of study completion ranged from 10.8% (4/37) to 40.3% (31/77) across academic medical centers, whereas results reporting on ClinicalTrials.gov ranged from 1.6% (2/122) to 40.7% (72/177).
Conclusions Despite the ethical mandate and expressed values and mission of academic institutions, there is poor performance and noticeable variation in the dissemination of clinical trial results across leading academic medical centers.
Introduction
Randomized clinical trials are the ideal means for evaluating the efficacy and safety of medical drugs and devices. Timely dissemination of the findings from clinical trials is a prerequisite for ensuring that clinical decisions made by patients and physicians reflect the best scientific evidence, and that future scientific investigation benefits from previous inquiry. Dissemination is principally achieved through publication in peer reviewed biomedical journals as well as through public reporting of results on clinical trial registries.1 2 3 4 However, a large body of research found that between 25% and 50% of clinical trials remain unpublished, sometimes years after study completion.5 6 7 8 9 Similarly, studies have shown that the results of many clinical trials are not reported promptly on ClinicalTrials.gov.10 11 12 13 14 15
Academic medical centers play a critical role in the clinical trials research enterprise. However, studies suggest that academically based investigators perform suboptimally in publishing8 16 and reporting trial results.14 15
We carried out a comprehensive examination of the rates of publication and reporting of results within two years for all completed clinical trials registered in ClinicalTrials.gov across more than 50 academic medical centers in the United States with active clinical research programs.
Methods
Data source and study sample
We used data from ClinicalTrials.gov through the Aggregate Analysis of ClinicalTrials.gov (AACT) database, reflecting data downloaded as of 27 September 2013, under the Clinical Trials Transformation Initiative. We identified all interventional clinical trials registered on ClinicalTrials.gov with a primary completion date (the date listed for when the trials finished collecting data for their primary endpoints) between October 2007 and September 2010 to ensure adequate time for publication in peer reviewed journals and the reporting of results. To identify trials with the responsible party based at an academic medical center, we selected those affiliated with an academic medical center, using the “role” field to identify the lead investigator, and his or her primary affiliation through the “affiliation” field. An academic medical center was one that included all hospitals owned or operated by the faculty and staff from a single academic institution and that was subject to the same institutional policies and review boards. We limited academic medical centers to those based in the United States, given that the US government created ClinicalTrials.gov as a database of trials that were either taking place or seeking approval for use in the United States.
Overall, at the time of our study 9620 unique affiliations were registered, sorted by frequency of occurrence. We used manual abstraction to determine if the affiliations could be identified as academic medical centers. After reviewing the preliminary data, we excluded academic medical centers with fewer than 40 clinical trials over the three year study period to ensure adequate sample size to determine rates of results reporting and publication. We identified and excluded clinical trials with an unknown or withdrawn status identified during the manual review process.
Publication rates
To determine the publication rates for clinical trials, six reviewers (RC, KHC, BW, KM, DL, AM) independently searched the biomedical literature between January and July 2014. We identified the earliest primary publication date of the main results of the trial, which reported the primary outcome. If there were multiple primary endpoints, we used the earliest publication that reported the results of at least one primary outcome.
We identified publications using a systematic three step search strategy. Firstly, we entered the national clinical trial identifier into ClinicalTrials.gov to find associated publications under the publications section and to identify the primary outcomes. Secondly, we searched the PubMed database using the national clinical trial identifier. Finally, if a matching publication was still not found, we searched the Scopus database (Elsevier, Philadelphia, PA) using the terms “[intervention name]” AND “clinical trial” in the “article title, abstract, keywords” field. If necessary, we added “[indication]” to the search. We chose Scopus for its extensive indexing, as it is one of the most comprehensive databases available and contains more than 50 million reported records from over 21 000 titles, including 100% coverage of articles indexed in Medline and PubMed.17
We used five criteria to identify matching publications: study design, indication, intervention, primary outcomes, and intention to treat enrollment. If we found multiple matching publications, we further refined the list by matching additional characteristics from the ClinicalTrials.gov registration, including primary investigators and study locations. We allowed enrollment to vary by up to 10% relative or 20 absolute if all other criteria matched.
We implemented several measures to ensure quality and consistency of the manual reviews. One lead investigator (RC) initially reviewed the first 50 trials of each assigned reviewer’s subsection with the assigned reviewer to ensure consistency in the search algorithm. A second reviewer independently reviewed any uncertainties during the manual review process, and the lead investigator resolved further conflicts by review. For validation, a team member not involved in the original reviews (ND) independently confirmed a 5% random sample of each team member’s collection of abstracted clinical trials (225 trials total), with discrepancies resolved by consensus. The random sample review confirmed the original reviewer’s findings in all but two instances (0.9%), which identified inconsistencies related only to the month of the primary publication. There were no instances where a reviewer reported no publication of the trial, but on random review a trial publication was discovered.
We then calculated time in months from the primary completion date to the date of primary publication, using the “primary completion date” field and the publication date. For publications available through online access before their official publication date, we used the earlier online access date as the publication date. Given our interest in examining timely dissemination of clinical trial results, we selected a publication timeframe of 24 months from study completion. Since dates are reported by the month and year only, we allowed for publication dates two years from the completion date up to the same month to effectively capture a window of less than 25 months. When assessing the rate of publication at each institution, we recognized clinical trials with a publication date before the primary completion date, but excluded these trials from analyses to determine the rate of publication within 24 months of study completion, as this was undefined (that is, <0).
Results reporting
We examined rates of results reporting on ClinicalTrials.gov as well as the time from study completion to results reporting for all clinical trials in our cohort. To determine whether results were reported within 24 months, we calculated the time in months from the “primary completion date” field to the “first received results date” field for each trial, then aggregated clinical trials by academic center. For publication rates, owing to dates only being reported by month, we effectively captured a window of less than 25 months. We included clinical trials with a results reporting date that preceded the primary completion date when assessing the rate of results reporting at each institution, but excluded the trials from analyses to examine the rate of results reporting within 24 months of study completion, as this was undefined (that is, <0).
Statistical analysis
For our aggregate sample of trials across all centers and within each center, we examined investigator reported baseline characteristics, including source of funding, area of study, features of the clinical trial design, and enrollment. For each center we calculated the proportion of trials that disseminated results, defined as either a primary publication or results reporting on ClinicalTrials.gov within 24 months of completion, as well as the overall rates of results reporting and publication at any time. In addition, we provided descriptive statistics on time to publication and results reporting for all trials with calculated times greater than 0 (that is, did not precede the primary completion date) for publication and results reporting. Statistical analyses were conducted with SAS version 9.3 (SAS Institute, Cary, NC).
Patient involvement
No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for recruitment, design, or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.
Results
We identified 5020 interventional clinical trials registered on ClinicalTrials.gov with a primary completion date between October 2007 and September 2010 and primarily affiliated with academic medical centers. Of these, we excluded 673 because the trial’s status was listed as unknown or withdrawn, leaving a study cohort of 4347 trials across 51 academic institutions. Oncology was the most common area of investigation, followed by behavior and mental disorders and cardiovascular diseases (table 1). Among the trials, 1005 (23%) enrolled more than 100 patients, 1216 (28%) were double blind, 2169 (50%) were phase II through IV trials, and 424 (9.8%) listed the National Institutes of Health as the primary sponsor (table 1).
Table 1.
Overall characteristics of clinical trials across top academic medical centers (≥40 trials), 2007-10. Values are numbers (percentages) unless stated otherwise
| Characteristics | Clinical trials |
| Total No of trials | 4347 |
| Total No of institutions | 51 |
| Conditions studied: | |
| Cancers and other neoplasms | 1458 (33.5) |
| Behavior and mental disorders | 553 (12.7) |
| Heart and blood diseases | 355 (8.2) |
| Nutritional and metabolic diseases | 288 (6.6) |
| Conditions of the urinary tract and sexual organs, and pregnancy | 226 (5.2) |
| Viral diseases | 222 (5.1) |
| Nervous system diseases | 220 (5.1) |
| Respiratory tract diseases | 173 (4.0) |
| Bacterial and fungal diseases | 69 (1.6) |
| Other | 783 (18.0) |
| Endpoint classification: | |
| Safety/efficacy | 1578 (36.3) |
| Efficacy | 1456 (33.5) |
| Safety | 248 (5.7) |
| Other | 177 (4.0) |
| Missing | 888 (20.4) |
| Study phase: | |
| 0 | 43 (1.0) |
| I | 608 (14.0) |
| I/II | 251 (5.8) |
| II | 1175 (27.0) |
| II/III | 106 (2.4) |
| III | 435 (10.0) |
| IV | 453 (10.4) |
| NA | 1276 (29.4) |
| No of study arms: | |
| 1 | 1698 (39.1) |
| 2 | 1780 (40.9) |
| 3 | 358 (8.2) |
| ≥4 | 269 (6.2) |
| Missing | 242 (5.6) |
| Intervention model: | |
| Single group | 1726 (39.7) |
| Parallel | 2000 (46.0) |
| Crossover | 290 (6.7) |
| Factorial | 130 (3.0) |
| Missing | 201 (4.6) |
| Masking: | |
| Open label | 2639 (60.7) |
| Single blind | 400 (9.2) |
| Double blind | 1216 (28.0) |
| Missing | 92 (2.1) |
| Randomization status: | |
| Randomized | 2453 (56.4) |
| Non-randomized | 884 (20.3) |
| Missing | 1010 (23.2) |
| Enrolled participants: | |
| <100 | 3102 (71.4) |
| 100-1000 | 1005 (23.1) |
| >1000 | 172 (4.0) |
| Missing | 68 (1.6) |
| Sponsor: | |
| National Institutes of Health | 424 (9.8) |
| Industry | 519 (11.9) |
| Other | 3404 (78.4) |
Dissemination of clinical trial results
Overall, 2892 of the 4347 clinical trials (66.5%) had been published or reported results as of July 2014. The time from primary completion to either publication or results reporting varied significantly, with the results of 1560 (35.9%) trials having been disseminated within 24 months and 1116 (25.7%) more than 24 months after primary study completion. In 216 (5.0%) trials, the publication date and/or results reporting date preceded the primary completion date; we excluded these from analyses of timing but counted them as having had results disseminated. Figure 1 shows the cumulative percentage of completed clinical trials with either published results or results reported on ClinicalTrials.gov over the study period.
Fig 1 Time to publication of results or results reporting for completed clinical trials across academic institutions. Of 4347 completed clinical trials, this figure excludes those without dissemination of results (n=1455) as well as those with publication date and results reporting date <0 (n=216)
Rates of publication of results or results reporting for completed clinical trials as well as the median time from study completion to dissemination varied considerably across academic institutions (table 2). The proportion of clinical trials with results disseminated within 24 months of completion ranged from 16.2% (6/37) to 55.3% (57/103) across academic institutions (fig 2). The overall rate of dissemination of clinical trial results across institutions ranged from 45.9% (17/37) to 76.7% (79/103). The median time from study completion to publication or results reporting on ClinicalTrials.gov ranged from 13.9 to 28.3 months.
Table 2.
Dissemination of clinical trial results across academic medical centers, 2007-10
| Institution | No of trials | Overall rate of results reported or publication No (%) | Median time from primary completion date to results reported or publication date (months)* | Rate of results reported or publication <24 months of study completion No (%) | Overall rate of publication No (%) | Median time from primary completion date to publication date (months)† | Rate of publication <24 months of study completion No (%) | Overall rate of results reporting on ClinicalTrials.gov No (%) | Median time from primary completion date to reporting of results (months)‡ | Trials with results reported <24 months No (%) | |||||||||||
| Baylor College of Medicine | 77 | 49 (63.6) | 20.4 | 28 (36.4) | 40 (51.9) | 20.4 | 25 (32.5) | 15 (19.5) | 34.7 | 6 ( 7.8) | |||||||||||
| Brigham and Women’s Hospital | 58 | 36 (62.1) | 19.7 | 22 (37.9) | 35 (60.3) | 21.5 | 20 (34.5) | 16 (27.6) | 19.7 | 10 (17.2) | |||||||||||
| Boston University | 42 | 32 (76.2) | 28.3 | 10 (23.8) | 26 (61.9) | 30.8 | 7 (16.7) | 14 (33.3) | 28.3 | 5 (11.9) | |||||||||||
| Case Western Reserve University | 68 | 49 (72.1) | 17.3 | 33 (48.5) | 43 (63.2) | 23.2 | 24 (35.3) | 21 (30.9) | 17.3 | 17 (25.0) | |||||||||||
| The Cleveland Clinic | 90 | 53 (58.9) | 18.4 | 30 (33.3) | 41 (45.6) | 18.4 | 25 (27.8) | 26 (28.9) | 33.0 | 9 (10.0) | |||||||||||
| University of Colorado | 59 | 41 (69.5) | 20.7 | 21 (35.6) | 38 (64.4) | 20.7 | 19 (32.2) | 10 (16.9) | 30.8 | 2 ( 3.4) | |||||||||||
| Columbia University | 113 | 72 (63.7) | 22.0 | 38 (33.6) | 65 (57.5) | 22.2 | 32 (28.3) | 30 (26.5) | 22.0 | 18 (15.9) | |||||||||||
| Cornell University | 36 | 18 (50.0) | 17.9 | 9 (25.0) | 14 (38.9) | 26.6 | 6 (16.7) | 6 (16.7) | 17.9 | 4 (11.1) | |||||||||||
| Dana-Farber Cancer Institute | 53 | 36 (67.9) | 25.5 | 14 (26.4) | 34 (64.2) | 25.5 | 14 (26.4) | 10 (18.9) | 36.1 | 2 ( 3.8) | |||||||||||
| Duke University | 187 | 141 (75.4) | 24.9 | 66 (35.3) | 116 (62.0) | 24.9 | 53 (28.3) | 75 (40.1) | 37.1 | 17 ( 9.1) | |||||||||||
| Emory University | 75 | 46 (61.3) | 22.2 | 22 (29.3) | 34 (45.3) | 22.2 | 18 (24.0) | 18 (24.0) | 28.5 | 5 ( 6.7) | |||||||||||
| Johns Hopkins University | 190 | 129 (67.9) | 24.3 | 59 (31.1) | 117 (61.6) | 24.3 | 51 (26.8) | 38 (20.0) | 33.6 | 14 ( 7.4) | |||||||||||
| Indiana University | 72 | 47 (65.3) | 15.5 | 32 (44.4) | 42 (58.3) | 15.5 | 26 (36.1) | 18 (25.0) | 18.7 | 13 (18.1) | |||||||||||
| University of Kansas | 36 | 21 (58.3) | 19.2 | 12 (33.3) | 18 (50.0) | 19.2 | 11 (30.6) | 8 (22.2) | 33.9 | 1 ( 2.8) | |||||||||||
| University of Maryland | 37 | 21 (56.8) | 18.3 | 14 (37.8) | 20 (54.1) | 18.3 | 13 (35.1) | 6 (16.2) | 29.1 | 3 ( 8.1) | |||||||||||
| Mayo Clinic | 268 | 201 (75.0) | 22.0 | 98 (36.6) | 174 (64.9) | 22.0 | 82 (30.6) | 96 (35.8) | 28.7 | 34 (12.7) | |||||||||||
| MD Anderson Cancer Center | 177 | 129 (72.9) | 16.8 | 88 (49.7) | 62 (35.0) | 22.0 | 26 (14.7) | 98 (55.4) | 16.8 | 72 (40.7) | |||||||||||
| Massachusetts General Hospital | 170 | 119 (70.0) | 25.2 | 65 (38.2) | 95 (55.9) | 25.2 | 46 (27.1) | 66 (38.8) | 27.4 | 29 (17.1) | |||||||||||
| University of Miami | 39 | 24 (61.5) | 22.5 | 11 (28.2) | 20 (51.3) | 22.5 | 10 (25.6) | 10 (25.6) | 33.5 | 3 ( 7.7) | |||||||||||
| Memorial Sloan-Kettering Cancer Center | 122 | 76 (62.3) | 19.0 | 37 (30.3) | 76 (62.3) | 19.0 | 35 (28.7) | 5 ( 4.1) | 42.9 | 2 ( 1.6) | |||||||||||
| Medical University of South Carolina | 52 | 35 (67.3) | 23.4 | 19 (36.5) | 33 (63.5) | 23.4 | 17 (32.7) | 14 (26.9) | 23.5 | 7 (13.5) | |||||||||||
| University of Nebraska | 37 | 17 (45.9) | 24.8 | 6 (16.2) | 13 (35.1) | 28.0 | 4 (10.8) | 6 (16.2) | 24.8 | 3 ( 8.1) | |||||||||||
| Northwestern University | 68 | 43 (63.2) | 18.9 | 29 (42.6) | 34 (50.0) | 18.9 | 23 (33.8) | 27 (39.7) | 25.1 | 13 (19.1) | |||||||||||
| Oregon Health and Science University | 73 | 50 (68.5) | 25.4 | 21 (28.8) | 26 (35.6) | 25.4 | 11 (15.1) | 37 (50.7) | 29.6 | 12 (16.4) | |||||||||||
| Ohio State University | 63 | 40 (63.5) | 18.7 | 25 (39.7) | 40 (63.5) | 18.7 | 25 (39.7) | 7 (11.1) | 26.9 | 3 ( 4.8) | |||||||||||
| University of Rochester | 60 | 46 (76.7) | 25.0 | 25 (41.7) | 38 (63.3) | 28.0 | 17 (28.3) | 27 (45.0) | 25.0 | 14 (23.3) | |||||||||||
| Mount Sinai School of Medicine | 55 | 32 (58.2) | 14.4 | 23 (41.8) | 28 (50.9) | 21.8 | 18 (32.7) | 5 ( 9.1) | 14.4 | 5 ( 9.1) | |||||||||||
| Stanford University | 131 | 71 (54.2) | 23.2 | 38 (29.0) | 65 (49.6) | 23.2 | 34 (26.0) | 10 ( 7.6) | 31.5 | 4 ( 3.1) | |||||||||||
| University of Alabama at Birmingham | 70 | 49 (70.0) | 23.7 | 27 (38.6) | 36 (51.4) | 23.7 | 19 (27.1) | 22 (31.4) | 23.7 | 13 (18.6) | |||||||||||
| University of California, Davis | 64 | 44 (68.8) | 22.0 | 23 (35.9) | 43 (67.2) | 22.0 | 22 (34.4) | 6 ( 9.4) | 46.7 | 1 ( 1.6) | |||||||||||
| University of Chicago | 76 | 50 (65.8) | 26.5 | 19 (25.0) | 45 (59.2) | 26.5 | 18 (23.7) | 14 (18.4) | 39.3 | 2 ( 2.6) | |||||||||||
| University of Cincinnati | 41 | 25 (61.0) | 25.1 | 9 (22.0) | 22 (53.7) | 25.1 | 8 (19.5) | 8 (19.5) | 30.6 | 3 ( 7.3) | |||||||||||
| University of California, Irvine | 42 | 26 (61.9) | 13.9 | 18 (42.9) | 21 (50.0) | 19.0 | 12 (28.6) | 14 (33.3) | 13.9 | 13 (31.0) | |||||||||||
| University of California, Los Angeles | 95 | 65 (68.4) | 25.0 | 31 (32.6) | 60 (63.2) | 25.0 | 29 (30.5) | 17 (17.9) | 30.7 | 6 ( 6.3) | |||||||||||
| University of California, San Diego | 75 | 47 (62.7) | 18.6 | 27 (36.0) | 44 (58.7) | 20.6 | 26 (34.7) | 14 (18.7) | 18.6 | 8 (10.7) | |||||||||||
| University of California, San Francisco | 133 | 99 (74.4) | 21.3 | 53 (39.8) | 84 (63.2) | 21.3 | 40 (30.1) | 52 (39.1) | 28.2 | 21 (15.8) | |||||||||||
| University of Florida | 55 | 42 (76.4) | 16.0 | 21 (38.2) | 34 (61.8) | 16.0 | 20 (36.4) | 22 (40.0) | 30.3 | 6 (10.9) | |||||||||||
| University of Michigan | 78 | 43 (55.1) | 22.5 | 20 (25.6) | 40 (51.3) | 22.5 | 19 (24.4) | 14 (17.9) | 31.4 | 4 ( 5.1) | |||||||||||
| University of Minnesota | 103 | 79 (76.7) | 15.8 | 57 (55.3) | 62 (60.2) | 21.0 | 37 (35.9) | 46 (44.7) | 15.8 | 35 (34.0) | |||||||||||
| University of North Carolina | 114 | 79 (69.3) | 24.3 | 42 (36.8) | 71 (62.3) | 24.3 | 34 (29.8) | 24 (21.1) | 25.8 | 12 (10.5) | |||||||||||
| University of Pennsylvania | 105 | 69 (65.7) | 21.5 | 39 (37.1) | 65 (61.9) | 22.9 | 35 (33.3) | 11 (10.5) | 21.5 | 7 ( 6.7) | |||||||||||
| University of Pittsburgh | 106 | 72 (67.9) | 21.6 | 39 (36.8) | 64 (60.4) | 22.3 | 34 (32.1) | 26 (24.5) | 21.6 | 14 (13.2) | |||||||||||
| University of Texas Houston | 36 | 23 (63.9) | 18.6 | 15 (41.7) | 21 (58.3) | 20.2 | 12 (33.3) | 8 (22.2) | 18.6 | 6 (16.7) | |||||||||||
| University of Texas Southwestern | 49 | 31 (63.3) | 22.8 | 17 (34.7) | 30 (61.2) | 22.8 | 16 (32.7) | 5 (10.2) | 26.5 | 2 ( 4.1) | |||||||||||
| University of Virginia | 32 | 21 (65.6) | 24.2 | 9 (28.1) | 19 (59.4) | 24.2 | 8 (25.0) | 5 (15.6) | 31.0 | 2 ( 6.3) | |||||||||||
| University of Washington | 141 | 77 (54.6) | 19.3 | 44 (31.2) | 66 (46.8) | 21.6 | 33 (23.4) | 26 (18.4) | 19.3 | 19 (13.5) | |||||||||||
| University of Wisconsin | 90 | 62 (68.9) | 22.6 | 32 (35.6) | 60 (66.7) | 22.6 | 30 (33.3) | 19 (21.1) | 28.1 | 6 ( 6.7) | |||||||||||
| Vanderbilt University | 110 | 70 (63.6) | 22.4 | 37 (33.6) | 57 (51.8) | 22.4 | 28 (25.5) | 36 (32.7) | 26.0 | 16 (14.5) | |||||||||||
| Wake Forest University | 53 | 34 (64.2) | 14.5 | 20 (37.7) | 26 (49.1) | 14.5 | 16 (30.2) | 14 (26.4) | 30.9 | 6 (11.3) | |||||||||||
| Washington University in St Louis | 94 | 55 (58.5) | 22.1 | 32 (34.0) | 51 (54.3) | 22.1 | 26 (27.7) | 19 (20.2) | 24.8 | 10 (10.6) | |||||||||||
| Yale University | 77 | 56 (72.7) | 23.0 | 34 (44.2) | 50 (64.9) | 23.0 | 31 (40.3) | 25 (32.5) | 34.4 | 8 (10.4) | |||||||||||
*Excludes trials without dissemination of results (n=1455) or a publication date or results reporting date that preceded the primary completion date (n=216).
†Excludes trials without a publication (n=1889) or with a publication date that preceded the primary completion date (n=261).
‡Excludes trials without results reported on ClinicalTrials.gov (n=3181) or with a results reporting date that preceded the primary completion date (n=2).
Fig 2 Rates of dissemination of clinical trial results (publication of results or reporting of results on ClinicalTrials.gov) within 24 months across academic institutions. Of 4347 completed clinical trials, this figure excludes trials without dissemination of results (n=1455) as well as those with publication date and results reporting date <0 (n=216)
Publication rates
Of the 4347 trials in our analysis, 2458 (56.5%) had been published as of July 2014. The time (months) from primary completion date to publication varied significantly, with 1245 (28.6%) having been published within two years and 952 (21.9%) more than 24 months after the primary completion date. Overall, 261 (6.0%) trials had a publication date that preceded the primary completion date; we excluded these from analyses of timing but counted them as having been published. The median publication time for these 2197 trials was 22.3 (interquartile range 14.0-33.0) months (see supplementary figure 1).
Rates of publication of results from completed clinical trials as well as median time from study completion to publication varied considerably across academic institutions (table 2). The proportion of clinical trials published within 24 months of study completion ranged from 10.8% (4/37) to 40.3% (31/77) across academic institutions. The overall rate of publication of clinical trial results ranged from 35.0% (13/37) to 67.2% (43/64) and the median time from study completion to initial publication of findings ranged from 14.5 to 30.8 months.
Results reporting
Of the 4347 completed clinical trials, 1166 (26.8%) reported results on ClinicalTrials.gov as of July 2014. The time (months) from primary completion date to results reporting varied significantly, with 547 (12.6%) trials reporting results within 24 months and 617 (14.2%) more than two years after the primary completion date. Two trials had a results reporting date that preceded the primary completion date; we excluded these from analyses of timing but counted them as reporting results. The median time from study completion to results reporting for these 1164 trials was 26.1 (interquartile range 16.4-36.6) months (see supplementary figure 2).
Rates of results reporting on ClinicalTrials.gov for completed clinical trials as well as the median time from study completion to results reporting varied considerably across academic institutions (table 2). The overall rate of results reporting of clinical trials ranged from 4.1% (5/122) to 55.4% (98/177). The median time from study completion to results reporting varied from 13.9 to 46.7 months, whereas the rate of results reporting within two years of study completion ranged from 1.6% (2/122) to 40.7% (72/177) across academic institutions.
Discussion
Our cross sectional examination of academic medical centers in the United States, including the nation’s most productive research institutions, showed poor performance for disseminating the results of completed clinical trials through publication in peer reviewed biomedical journals or reporting of results on ClinicalTrials.gov. Only 29% (1245/4347) of completed clinical trials conducted by the faculty at major academic institutions were published within two years of study completion and only 13% (547/4347) reported results on ClinicalTrials.gov. Our study revealed marked variation in rates of dissemination of clinical trial results across academic institutions, with more than a twofold variation in the median time from study completion to dissemination of results and more than a threefold variation in the rate of dissemination across institutions. However, no academic center published more than 40% of completed clinical trials within two years of completion or reported results for more than 41% of its trials.
Randomized clinical trials are the ideal means for evaluating the efficacy and safety of a drug or device. Timely dissemination of the findings of clinical trials is not only essential to support evidence based decision making by patients and providers, but is required to fulfill the ethical obligation that investigators and sponsors have to study participants, professional values, and the mission of academic medical centers. Recent work has examined issues of data sharing more broadly, including the questions of which data would be made available, to whom, when, and under whose oversight.1 2 3 4 18 Our analysis represents the first systematic examination of the publication of clinical trials and reporting rates for results across academic centers. Though the Institute of Medicine,19 the National Institutes of Health,20 the European Medicines Agency,21 and the World Health Organization22 have helped spur the discussion about expanding the frontiers of data transparency, our findings suggest that far more basic elements of transparency in the clinical trial enterprise—the need to publish findings and report results—remain elusive.
While seemingly axiomatic that the results of clinical trials led by the faculty at leading academic institutions will undergo peer reviewed publication, our study found that 44% of such trials have not been published more than three, and up to seven, years after study completion. This is consistent with previous studies that have reported between 25% and 50% of clinical trials remain unpublished as much as several years after completion.5 6 7 9 A recent examination of the publication of clinical trials funded by the National Institutes of Health found that 54% were unpublished within 30 months of trial completion and, among those published, the median time to publication was 23 (interquartile range 14-36) months.8 Similarly, an analysis of 244 extramural clinical trials supported by the National Heart, Lung, and Blood Institute and completed between 1 January 2000 and 31 December 2011 found that 43% of clinical trials remained unpublished within 30 months.16 Our analysis extends previous work by reporting publication rates across leading academic institutions and shows a nearly twofold variation in the rate of publication overall and a more than threefold variation in the rate of publication within 24 months.
With regard to reporting of results on ClinicalTrials.gov, we found that only 13% of trials that were registered, completed, and led by the faculty of an academic medical center reported results within 24 months of completion. Previous studies have documented suboptimal rates of results reporting,10 11 12 13 14 and a recent analysis of clinical trials subject to the Food and Drug Administration Amendments Act23 mandate to report results within 12 months of study completion found that 13% complied with the legislative requirement.15 Notably, trials funded by the National Institutes of Health and other government or academic institutions were significantly less likely to adhere to the FDA Amendments Act mandate than were trials supported by industry. Our analysis extends this work by highlighting noticeable variation in rates of results reporting within 24 months of study completion across academic medical centers, ranging from 2% to 41%.
The results reporting imperative emanates from widespread concern about selective publication of clinical trial results. Despite ethical obligations to participants, the values espoused by academic centers, and in some instances statutory requirements, there is no effective enforcement mechanism and no repercussions to academic institutions or individual investigators for failing to meet them. The National Institutes of Health recently announced a proposed policy in which “timely reporting of clinical trials will be taken into consideration during review of subsequent applications for funding,” but the consequence of non-reporting remains unspecified and the policy is currently undergoing public comment.20
Limitations of this study
Our analysis has several limitations. Firstly, in some clinical trials the reporting of results and publication dates preceded the primary completion date, likely reflecting errors in data entry. We gave institutions credit for all publications, even those with publication dates that preceded the primary completion date. Though we did not include these clinical trials in analyses of time from study completion to reporting of results or publication, any bias in our results would favor academic centers. In addition, this raises the need for ClinicalTrials.gov to develop oversight mechanisms and built-in tools that could help prevent these types of errors. Secondly, our analyses included clinical trials completed through September 2010, because adequate time was required to assess rates of publication and results reporting of completed clinical trials. Ongoing study is required to determine if academic health centers have subsequently improved reporting and publication of clinical trial results. Given the large number of trials analyzed, we did not contact trial investigators. Notably, phase I trials do not need to be registered or the results reported on ClinicalTrials.gov. However, the issue of non-publication and lack of dissemination of results remains prevalent across all types of trials. Irrespective of type, there remains a need and importance to share all trial results with the academic community and advance the scientific process.
Conclusion
We found noticeable variation and poor performance across leading academic medical centers in the dissemination of clinical trial results. The lack of timely reporting and publication fundamentally impairs the research enterprise, violates the commitment made by investigators to patients and funders, squanders precious time and resources, and threatens to compromise evidence based clinical decision making.
What is already known on this topic
Timely dissemination of clinical trial results is required to honor the commitment of study participants, advance the research enterprise, and improve clinical care, but little is known about the performance of academic medical centers in this endeavor
Previous limited studies have shown that between 25% and 50% of clinical trials remain unpublished, sometimes years after completion, and the performance of academically based investigators in publishing and reporting of trial results is suboptimal
What this study adds
Academic medical centers showed noticeable variation and poor performance in the dissemination of clinical trial results
Only 29% of completed clinical trials conducted by the faculty at major academic centers were published within two years of completion and only 13% reported results on ClinicalTrials.gov
Additional tools and mechanisms are needed to rectify this lack of timely reporting and publication, as they impair the research enterprise and threaten to undermine evidence based clinical decision making
Web extra.
Web extra material supplied by the author
Supplemental appendix: additional information
During the conduct of this work, RC, KHC, BW, and DYL were affiliated with the Department of Internal Medicine, Yale School of Medicine, and KM was affiliated with the Center for Outcomes Research and Evaluation, Yale-New Haven Hospital.
Contributors: RC, NRD, JSR, WZ, and HMK conceived and designed the study. RC, NRD, JSR, KHC, BW, KM, DYL, and AM extracted data and performed the manual review. WZ performed the statistical analyses. RC and NRD drafted the manuscript. All authors interpreted the data, critically revised the manuscript for important intellectual content, and approved the final manuscript. JSR and HMK provided administrative, technical, or material support. RC and NRD are the joint first authors and guarantors.
Funding: This study received no specific funding or grant from any agency in the public, commercial, or not for profit sectors. NRD is supported by the Agency for Healthcare Research and Quality (grant K12 HS023000-01) supports NRD. JSR is supported by the National Institute on Aging (grant K08 AG032886) and by the American Federation for Aging Research through the Paul B Beeson career development award programme. HMK is supported by the National Heart, Lung, and Blood Institute (grant U01 HL105270-05, Center for Cardiovascular Outcomes Research at Yale University). These sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: NRD, JSR, and HMK are recipients of a research agreement from Johnson and Johnson (Janssen), through Yale University, to develop methods of clinical trial data sharing, and of contracts from the Centers for Medicare & Medicaid Services to develop and maintain performance measures that are used for public reporting. JSR and HMK are recipients of a research agreement from Medtronic, through Yale University, to develop methods of clinical trial data sharing, and of a grant from the Food and Drug Administration to develop methods for post-market surveillance of medical devices. HMK chairs a cardiac scientific advisory board for UnitedHealth.
Ethical approval: Not required.
Data sharing: Requests for the statistical code and dataset can be made to the corresponding author (harlan.krumholz@yale.edu). The dataset will be made available through a publicly accessible repository on publication, at the Dryad Digital Repository (datadryad.org).
Transparency: The lead authors (RC, NRD) affirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
References
- 1.Ross JS, Krumholz HM. Ushering in a new era of open science through data sharing: the wall must come down. JAMA 2013;309:1355-6. 10.1001/jama.2013.1299. 23508736. [DOI] [PubMed] [Google Scholar]
- 2.Krumholz HM, Ross JS, Gross CP, et al. A historic moment for open science: the Yale University Open Data Access project and medtronic. Ann Intern Med 2013;158:910-1. 10.7326/0003-4819-158-12-201306180-00009. 23778908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Krumholz HM. Open science and data sharing in clinical research: basing informed decisions on the totality of the evidence. Circ Cardiovasc Qual Outcomes 2012;5:141-2. 10.1161/CIRCOUTCOMES.112.965848. 22438459. [DOI] [PubMed] [Google Scholar]
- 4.Ross JS, Lehman R, Gross CP. The importance of clinical trial data sharing: toward more open science. Circ Cardiovasc Qual Outcomes 2012;5:238-40. 10.1161/CIRCOUTCOMES.112.965798. 22438465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:281-6. 10.1001/jama.279.4.281. 9450711. [DOI] [PubMed] [Google Scholar]
- 6.Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:867-72. 10.1016/0140-6736(91)90201-Y. 1672966. [DOI] [PubMed] [Google Scholar]
- 7.Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 2009;6:e1000144. 10.1371/journal.pmed.1000144. 19901971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ 2012;344:d7292. 10.1136/bmj.d7292. 22214755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ross JS, Mocanu M, Lampropulos JF, Tse T, Krumholz HM. Time to publication among completed clinical trials. JAMA Intern Med 2013;173:825-8. 10.1001/jamainternmed.2013.136. 23460252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gopal RK, Yamashita TE, Prochazka AV. Research without results: inadequate public reporting of clinical trial results. Contemp Clin Trials 2012;33:486-91. 10.1016/j.cct.2012.02.001. 22342449. [DOI] [PubMed] [Google Scholar]
- 11.Law MR, Kawasumi Y, Morgan SG. Despite law, fewer than one in eight completed studies of drugs and biologics are reported on time on ClinicalTrials.gov. Health Aff (Millwood) 2011;30:2338-45. 10.1377/hlthaff.2011.0172. 22147862. [DOI] [PubMed] [Google Scholar]
- 12.Gill CJ. How often do US-based human subjects research studies register on time, and how often do they post their results? A statistical analysis of the Clinicaltrials.gov database. BMJ Open 2012;2:e001186. 10.1136/bmjopen-2012-001186. 22936820. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Saito H, Gill CJ. How frequently do the results from completed US clinical trials enter the public domain?--A statistical analysis of the ClinicalTrials.gov database. PLoS One 2014;9:e101826. 10.1371/journal.pone.0101826. 25025477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012;344:d7373. 10.1136/bmj.d7373. 22214756. [DOI] [PubMed] [Google Scholar]
- 15.Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med 2015;372:1031-9. 10.1056/NEJMsa1409364. 25760355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926-34. 10.1056/NEJMsa1300237. 24224625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Scopus. Elsevier, 2014. https://www.elsevier.com/solutions/scopus.
- 18.Spertus JA. The double-edged sword of open access to research data. Circ Cardiovasc Qual Outcomes 2012;5:143-4. 10.1161/CIRCOUTCOMES.112.965814. 22438460. [DOI] [PubMed] [Google Scholar]
- 19.Lo B. Sharing clinical trial data: maximizing benefits, minimizing risk. JAMA 2015;313:793-4. 10.1001/jama.2015.292. 25594500. [DOI] [PubMed] [Google Scholar]
- 20.Hudson KL, Collins FS. Sharing and reporting the results of clinical trials. JAMA 2015;313:355-6. 10.1001/jama.2014.10716. 25408371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Groves T. Big strides in Europe towards clinical trial transparency. BMJ 2014;349:g6276. 10.1136/bmj.g6276. 25331626. [DOI] [PubMed] [Google Scholar]
- 22.Moorthy VS, Karam G, Vannice KS, Kieny MP. Rationale for WHO’s new position calling for prompt reporting and public disclosure of interventional clinical trial results. PLoS Med 2015;12:e1001819. 10.1371/journal.pmed.1001819. 25874642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Food and Drug Administration Amendments Act of 2007. (Public Law No. 110-85; Section 801; 2007.)
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental appendix: additional information