Abstract
Objectives
To examine 1) the publication rate of registered otology trials in ClinicalTrials.gov, 2) the public availability of the results, 3) the study characteristics associated with publication, and 4) the time to publication after trial completion.
Background
Publication bias, the publication or non-publication of research findings, depending on the nature and direction of results, is accountable for wrong treatment decisions. The extent of publication bias in otology trials has not been evaluated.
Methods
All registered otology trials were extracted from ClinicalTrials.gov with completion date up to December 2015. A search strategy was used to identify corresponding publications up to June 2017, providing at least 18 months to publish the results after trial completion. Characteristics were obtained from ClinicalTrials.gov and corresponding publications. Regression models were used to examine study characteristics associated with publication or non-publication.
Results
From the 419 trials identified on ClinicalTrials.gov, 225 (53.7%) corresponding publications were found in PubMed. Among these, 109 (48.4%) publications were cited on ClinicalTrials.gov and 124 (55.1%) articles reported the National Clinical Trial registry number. For 36 (8.6%) trials, results were only reported in ClinicalTrials.gov. Trials with a biological intervention were more likely to be published than studies involving drugs (odds ratio (OR) 10.41, 95% confidence interval (CI) 1.26–86.22, P = 0.030). Trials funded by industry were less likely to be published (OR 0.46, CI 0.25–0.84, P = 0.011). The median trial duration was 20 months (interquartile range (IQR) 26 months), and median time from trial completion to publication was 24 months (IQR 22 months).
Conclusion
In 37.7% of the registered otology trials the results remained unpublished, even several years after trial completion. With little citations on ClinicalTrials.gov and low reporting of the Clinical Trial registry number, the accessibility is limited. Our findings show that there is room for improvement in accuracy of trial registration and publication of results, in order to diminish publication bias in otology studies.
Introduction
In an era of evidence based medicine (EBM), clinicians must have access to clear, transparent, and complete information in scientific publications in order to draw conclusions. In 2009, Chalmers and Glasziou made the claim that as much as 85% of research investment is wasted due to a cumulative effect, including the production of biased reports.[1]
Publication bias is the publication or non-publication of research findings, depending on the nature and direction of the results.[2] Selective publication of studies, including delayed publication of completed trials or non-publication, may lead to incomplete or misrepresented evidence, impairing evidence-based clinical practice, undermining guideline recommendations, and can thereby make clinical practice and patient care less effective.[3,4] Previous studies reported that in 23–68% of the registered trials, the results remain unpublished, even several years after trial completion.[3,5–14] Trials that produce negative, inconclusive or non-significant results are less likely to be published or are published significantly later than their positive counterparts.[5,15,16]
Efforts are undertaken to diminish publication bias. First, in 2000, clinical trial registries, such as ClinicalTrials.gov, were established.[5] These registries have the potential to address selective publication by publicly cataloguing clinical trials and promoting trial transparency and accountability.[3] All registered trials receive a unique clinical trial registry number, for example the National Clinical Trial (NCT) number for trials registered in ClinicalTrials.gov. If this number is reported in the article, registered trial information is easy to find.
Second, since 2005, the International Committee of Medical Journal Editors (ICMJE) requires registration of both observational and interventional studies before enrolment of the first patient in a public database as a condition for publication.[17] Consequently, the United States Congress expanded this requirement, only for interventional studies assessing medicines and medical devices, by passing the U.S. Food and Drug Administration Amendments Act (FDAAA), section 801. This mandate requires sponsors or principal investigators of applicable clinical trials to submit results no later than 12 months after completion date, unless legally acceptable reasons for delay are evident. Unfortunately, studies have shown that compliance with the FDAAA 801 provisions is generally poor.[18–20]
Third, from January 2017, the National Institutes of Health (NIH) regulations require that all clinical trials, funded in whole or in part by the NIH, are registered in ClinicalTrials.gov within 21 days after enrolment of the first participant. The results from these trials have to be submitted to ClinicalTrials.gov within one year after trial’s primary completion date.[21] In the European Union, the Clinical Trials Regulation No. 536/2014 will replace the existing Clinical Trials Directive No. 2001/20/EC. Both require trial registration prior to its start for clinical trials with medicinal products. A summary of the results must be submitted within one year from the end of the clinical trial, irrespective of the outcome. The Regulation will harmonise the assessment and supervision processes for clinical trials throughout the EU, and is currently scheduled to entry into force in 2019.[22,23] Finally, the Declaration of Helsinki requires all researchers, authors, sponsors, editors and publishers to have ethical obligations with regard to the publication and dissemination of the results of all research.[24]
In the field of otology, the extent of publication bias is still unknown. Therefore, we aim to examine 1) the rate of publication of registered trials in ClinicalTrials.gov, 2) the public availability of the results, including reporting of trial registration numbers, 3) the study characteristics associated with publication, and 4) the time to publication after trial completion.
Methods
Search for eligible trials at ClinicalTrials.gov
Trials were obtained from ClinicalTrials.gov on January 10, 2017 by one investigator (JvH). The topic ‘Ear, Nose and Throat Diseases’ was selected and all available information of all registered otorhinolaryngology trials was obtained and exported into a spreadsheet. Subsequently trials with otology topics were extracted, including neuro-otology.
We excluded observational studies, duplicate labelled trials (the same NCT number registered within different topics, e.g. vestibular disease, hearing loss, deafness, etc.), as well as trials with a primary completion date after December 31, 2015, in order to provide at least a one year period in which investigators could submit their results, consistent with FDAAA legislation. All trial statuses (e.g. ‘Completed’, ‘Terminated’, ‘Recruiting’, ‘Unknown’, etc.) and identical trials registered with different NCT numbers were included (e.g. the same trial registered separately by the sponsor and by the principal investigator).
Search strategy for publication of trial outcomes
Two investigators (JvH and IvB) searched for publications of the eligible trials independently using a standardized strategy (Fig 1). If there was any inconsistency, a third investigator (IS) reviewed the data and consensus was reached after discussion. Publications, defined as a complete manuscript in a peer-reviewed journal, were selected up to June 30th 2017, allowing a minimum time of 18 months for the investigators to submit an article for publication, peer review and editorial processes.
Fig 1. Flowchart of our search strategy and analysis.
Our search yielded a total of 3,727 registered otorhinolaryngology trials, of which 3,308 were excluded. In total 419 eligible otology trials were identified.
The ‘more information’ field within ClinicalTrials.gov was examined to determine if the trial sponsor or investigator provided a citation of an article describing the corresponding trial, as this field is used to disclose citations of trial results or other relevant research.
If no citation was provided, PubMed was used to search the Medical Literature Analysis and Retrieval System Online (MEDLINE) database for publications. An extensive search was performed using the NCT number, names of (principal) investigators, affiliations/institutions or keywords, e.g. trial title, intervention, primary and secondary outcome measures, study design, etc.
Publications (including preliminary results and pilot studies) of corresponding trials were marked as ‘publication’. If no corresponding article was found, we marked it as ‘no publication’. When results were only reported in ClinicalTrials.gov, we marked it as ‘results reported in ClinicalTrials.gov’. Research protocols and publications that provided the corresponding NCT number, but did not report outcomes of the trial, were excluded. If multiple corresponding publications were found for one trial, the article reporting primary outcome measures was chosen in order to calculate the most accurate time to publication.
Assessment of independent variables
The following data were extracted from ClinicalTrials.gov: NCT number, recruitment status, type of intervention (e.g. drugs, biological interventions, medical devices, or non-invasive interventions), age of participants, study phase (stages of clinical trials studying a drug or biological product, based on definitions developed by the U.S. Food and Drug Administration (FDA)), study topic, funding, number of patients enrolled, start date, ‘Primary completion date’, ‘Completion date’, and the date ‘Results first received’. In ClinicalTrials.gov the ‘Primary completion date’ is defined as the date that the final participant was examined for the purposes of final collection of data for the primary outcome. The ‘Completion date’ is defined as the date the final participant was examined for the primary and secondary outcome measures, and adverse events.[25] We considered a trial as ‘completed’, if it had one of the following recruitment statuses in ClinicalTrials.gov: ‘Completed, has results’, ‘Completed, no results available’, or ‘Terminated’.
The trial duration was calculated as the number of months from start date to completion date. In our study, the ‘Primary completion date’ was used for this calculation. When this was not provided, the ‘Completion date’ was used. The topic of the selected trials was evaluated by screening the title and the ‘study details’ section in ClinicalTrials.gov.
Assessment of published articles
Reporting of the NCT number and month and year of publication were noted. Time to publication for published trials was calculated as the number of months from the completion date of the trial to the publication date of the article. The electronic publication date or, when this date was not provided, the publication date in print was used to determine the publication date. Time to publication of results on ClinicalTrials.gov was calculated as the number of months from the completion date of the trial to the date the results were first received.
Data analysis
Categorizations of data elements were made, with the exception of two continuous variables (patient enrolment and trial duration). The Shapiro-Wilk test was used to test for normal distribution. Patient enrolment and trial duration were not normally distributed.
Age was a continuous variable, which we dichotomized to ‘including children’ (any participants with the age of 0 to 18 years old), or ‘not including children’ (any participants with the age of 18 years old and above). ClinicalTrials.gov classifies funding as follows; industry, NIH, federal government of the United States (U.S. federal), and other (including hospitals, universities, and non-profit organizations). For analysis, we categorized funding to ‘industry’, ‘non-industry’ (including NIH, U.S. federal, and other organizations) or ‘both’, in case several sponsors of multiple funding sources were listed. We categorized interventions to ‘drug’, ‘biological’, ‘device’, and ‘other’ (e.g. behavioral, dietary, genetic, procedure, radiation, and other interventions). When study phase was ‘Not Applicable’ (trials without FDA-defined phases, including trials of devices or behavioral interventions), we classified it as ‘missing’. We dichotomized study topic to ‘tinnitus’ (tinnitus as ‘condition or disease’ with tinnitus outcome measures), or ‘other’ (‘condition or disease’ not including tinnitus). Year of study completion was a continuous variable, which we categorized to ‘before 2006’, ‘2006–2007’, and ‘after 2007’, in order to evaluate the effect of ICMJE requirements (2005) and the FDAAA 801 (2007).
We used descriptive analyses to assess the differences between published and unpublished trials, and time to publication. For the completed trials, the number (percent) for discrete variables and median (interquartile range, IQR) for continuous variables were calculated.
We assessed the possible reasons for publication with binary logistic regression. Univariate regression was used to determine the explanatory variables for the multivariate logistic regression model. Solely variables with p-values less than 0.10 in the univariate level were used. We reported odds ratios (OR) along with 95% confidence intervals (CI) to determine statistical significance for the multivariate regression analysis. We consider a 95% CI not containing 1.00 as statistically significant. In case of discrepancies between p-values and 95% confidence intervals, the confidence interval is leading in the statistical significance. We checked the explanatory variables for multicollinearity. We used IBM SPSS statistics for Windows, version 25.0 (IBM Corp., Armonk, NY, USA) for analysis.
Results
Accessibility of results of registered otology trials
Our electronic trial registry search in ClinicalTrials.gov yielded a total of 3,727 registered otorhinolaryngology trials, of which 3,308 were excluded (Fig 1). In total 419 eligible otology trials, with start dates from July 1992 to July 2015, were identified (S1 Dataset). The median trial duration was 20 months (IQR 26 months). We found 225 (53.7%) corresponding articles; 109 (48.4%) were reported in ClinicalTrials.gov, and 116 (51.6%) were found on PubMed by keywords (S1 Table and Fig 2). Among all the publications found, 124 (55.1%) reported the NCT number in the article. For 90 (21.5%) of the 419 trials, results were provided in the ClinicalTrials.gov results database. For 36 (40.0%) of these 90 trials, no corresponding peer reviewed articles were found in PubMed. When considering publications and the results reported in ClinicalTrials.gov together, the rate of publication increased from 53.7% to 62.3%. So, in 37.7% of the registered otology trials the results remained unpublished.
Fig 2. Recruitment status and rate of publication of all otology trials registered in ClinicalTrials.gov.
Unknown status: a study on ClinicalTrials.gov which last known status was ‘recruiting’, ‘not yet recruiting’, or ‘active, not recruiting’ but that has passed its completion date, and the status has not been last verified within the past 2 years. Terminated: the study has stopped early and will not start again. Other: trials with recruitment status ‘withdrawn’, ‘suspended’, ‘enrolling by invitation’, ‘recruiting’ and ‘(active,) not yet recruiting’. For all data, see S1 Table.
Among the 419 eligible trials, 341 (81.4%) trials reported the recruitment status as ‘Completed’ or ‘Terminated’, of which 203 (59.5%) corresponding articles were found. For the remaining 78 (18.6%) trials (e.g. recruitment status ‘Unknown’, ‘Recruiting’, etc.), 22 (28.2%) corresponding articles were found by our extensive search.
Study characteristics associated with publication of completed trials
Table 1 shows characteristics of completed trials, according to publication status. Trials with a biological intervention were more likely to be published than studies involving drugs (OR 10.41, 95% CI 1.26–86.22, P = 0.030). Trials funded by industry were less likely to be published (OR 0.46, CI 0.25–0.84, P = 0.011). Whether or not children were included, study phase, study topic, year of completion, trial duration, and patient enrolment showed no association with publication (P > 0.05). We found no multicollinearity (Variance Inflation Factor 1.001–1.365) among the explanatory variables.
Table 1. Binary logistic regression for effect of different characteristics on publication of completed otology trials registered in ClinicalTrials.gov.
| Characteristics | Total (n = 341) | Publication (n = 203) | No publication (n = 138) | Univariate | Multivariate | |||
|---|---|---|---|---|---|---|---|---|
| n (%) | n (%) | n (%) | OR (95% CI) | p-value | OR (95% CI) | p-value | ||
| Interventions | Drug | 134 (39.3) | 64 (31.5) | 70 (50.7) | Reference | Reference | ||
| Biological | 14 (4.1) | 13 (6.4) | 1 (0.7) | 14.22 (1.81–111.78) | 0.012 | 10.41 (1.26–86.22) | 0.030 | |
| Device | 76 (22.3) | 44 (21.7) | 32 (23.2) | 1.50 (0.85–2.65) | 0.159 | 1.64 (0.89–3.03) | 0.117 | |
| Othera | 116 (34.0) | 81 (39.9) | 35 (25.4) | 2.53 (1.50–4.27) | 0.000 | 1.73 (0.90–3.33) | 0.098 | |
| Missing | 1 (0.3) | 1 (0.5) | 0 (0.0) | |||||
| Age | Including children | 129 (37.8) | 75 (36.9) | 54 (39.1) | 0.91 (0.58–1.42) | 0.683 | ||
| Not including children | 212 (62.2) | 128 (63.1) | 84 (60.9) | Reference | ||||
| Study phase | Phase 0 | 6 (1.8) | 3 (1.5) | 3 (2.2) | Reference | |||
| Phase 1 | 15 (4.4) | 9 (4.4) | 6 (4.3) | 1.50 (0.22–10.07) | 0.677 | |||
| Phase 1|2 | 13 (3.8) | 10 (4.9) | 3 (2.2) | 3.33 (0.43–26.04) | 0.251 | |||
| Phase 2 | 59 (17.3) | 36 (17.7) | 23 (16.7) | 1.57 (0.29–8.43) | 0.602 | |||
| Phase 2|3 | 12 (3.5) | 7 (3.4) | 5 (3.6) | 1.40 (0.20–10.03) | 0.738 | |||
| Phase 3 | 62 (18.2) | 29 (14.3) | 33 (23.9) | 0.88 (0.16–4.70) | 0.880 | |||
| Phase 4 | 28 (8.2) | 19 (9.4) | 9 (6.5) | 2.11 (0.35–12.60) | 0.412 | |||
| Missing | 146 (42.8) | 90 (44.3) | 56 (40.6) | |||||
| Study topic | Tinnitus | 80 (23.5) | 53 (26.1) | 27 (19.6) | 1.45 (0.86–2.45) | 0.163 | ||
| Other | 261 (76.5) | 150 (73.9) | 111 (80.4) | Reference | ||||
| Year of study completion | Before 2006 | 37 (10.9) | 29 (14.3) | 8 (5.8) | Reference | Reference | ||
| 2006–2007 | 18 (5.3) | 13 (6.4) | 5 (3.6) | 0.72 (0.20–2.62) | 0.615 | 1.50 (0.37–6.11) | 0.571 | |
| After 2007 | 280 (82.1) | 158 (77.8) | 122 (88.4) | 0.36 (0.16–0.81) | 0.014 | 0.47 (0.19–1.17) | 0.106 | |
| Missing | 6 (1.8) | 3 (1.5) | 3 (2.2) | |||||
| Funding | Industry | 120 (35.2) | 53 (26.1) | 67 (48.6) | 0.35 (0.22–0.56) | 0.000 | 0.46 (0.25–0.84) | 0.011 |
| Non-industry | 196 (57.5) | 136 (67.0) | 60 (43.5) | Reference | Reference | |||
| Both | 25 (7.3) | 14 (6.9) | 11 (8.0) | 0.56 (0.24–1.31) | 0.181 | 0.70 (0.27–1.83) | 0.461 | |
| Trial durationb | Median (IQR) | 19.5 (25.75) | 23.00 (27) | 17.00 (23) | 1.03 (1.01–1.04) | 0.000 | 1.02 (1.00–1.03) | 0.021 |
| Missing, n (%) | 9 (2.6) | 9 (50.0) | 9 (50.0) | |||||
| Patient enrolmentc | Median (IQR) | 65 (147) | 66 (172) | 70 (154) | 1.00 (1.00–1.00) | 0.253 | ||
| Missing, n (%) | 6 (1.8) | 9 (69.2) | 4 (30.8) | |||||
a Behavioral, dietary, genetic, procedure, radiation, and other interventions.
b Calculated as the interval between the start and completion dates, in months.
c Number of patients enrolled in trial.
CI = Confidence Interval; IQR = interquartile range; OR = Odds Ratio
Bold values are statistically significant.
Time to publication, rate of publication and trial completion over time
The median time from study completion to publication was 24 months (IQR 22 months, n = 200). Among these trials, 21.0% was published within 12 months after study completion. For 90 (21.5%) of the 419 eligible trials, results were reported in ClinicalTrials.gov. For these trials, median time from study completion to ‘results first received’ was 22.5 months (IQR 29 months), and in 27.8% results were published within 12 months after study completion. Fig 3 shows the rate of publication for each year of trial completion. Fig 4 shows a decline in time to publication between 2001 and 2005, which remained constant after 2005.
Fig 3. Rate of publication after trial completion over time.
Results are shown for 335 completed trials. For six trials, no (primary) completion dates were reported. n = number of completed trials.
Fig 4. Time to publication after trial completion over time.
Results are shown for 200 publications of completed trials. For three published trials, no (primary) completion dates were reported. *Publication before completion date. n = number of completed trials.
Discussion
Summary of results
This is the first study to assess trial registration, publication rate and characteristics of otology trials. It demonstrated a considerable source of publication bias. In 37.7% of the registered otology trials the results remained unpublished, even several years after trial completion. When publications were found, the majority of the publications was not cited on ClinicalTrials.gov, and a substantial proportion of the publications did not report the NCT number in the article, which would have made it easier for clinicians to access results.
Comparison with literature
Studies assessing trial registration and publication rate within other fields like rheumatoid arthritis[5], macular degeneration[8], and trials on diseases in the digestive system[7] showed similar outcomes with publication rates of respectively 64.5%, 54% and 63%. Moreover, consistent with other research[5,10,26], our study showed a marginal percentage of completed trials that submitted results in ClinicalTrials.gov. Although trial registry has the potential to improve transparency[27] and to address selective reporting[3], its usefulness is limited by the lack of absolute requirements to complete and update optional data elements, and suboptimal study reporting, as for example the non-reporting of NCT numbers.
Several studies, including ours, showed lower publication rates in trials funded by industry[3,10,11,13], whereas other studies, found no significant association.[5,8,14,28,29] Consistent with our study, a study assessing 143 randomized controlled trials concerning rheumatoid arthritis showed no increase of rate of publication after the ICMJE requirements (2005) or the FDAAA (2007).[5] In contrast to our findings, a cross sectional study of 818 registered surgical randomized controlled trials showed no association of publication with intervention type.[13]
We found a median time to publication of 24 months (IQR 22 months). Consistent with our data, there has been an improvement over the years of publication time reported in literature.[5–7,27] However, we did not take the different lengths of follow-up into account by using survival analysis. This could explain the slight decrease of time to publication after 2014. Therefore, grounded statements on time to publication assessed over time cannot be made.
Studies assessing publication of registered trials did not take every recruitment status into account. It is remarkable that still in 28.2% of trials with recruitment status ‘Unknown’, ‘Recruiting’, ‘Enrolling’, etc. results were published. The majority of these trials had the recruitment status ‘Unknown’: a trial registration that has not been verified by the sponsor or investigator within the past 2 years. This highlights the lack of accurate data submission and registration in ClinicalTrials.gov. In addition, the completion status in trial registries is commonly incorrect.[30] Likewise, we found deficits in registration records of ClinicalTrials.gov; missing and inaccurate data elements, deviations in trial registrations, and deviations in labelling of trial topics.
Methodological considerations
Our study is the first to assess the trial registration, publication rate, and characteristics of otology trials. We considered all otology trials registered in the largest registry, ClinicalTrials.gov, including trials registered from across the world, operated by the National Library of Medicine at the NIH.[31,32] It permits a detailed assessment of exactly where in the research pathway publication bias can be targeted. We acknowledge that more trials could have been included when other trial registries were used. However, ClinicalTrials.gov provided a representative sample, as it is the largest trial registry by far[33], and thus the publication rate is not likely to be different for other trial registries.
We must consider some limitations of our study. First, only PubMed was searched for publications, which might have induced some omissions, underestimating the rate of publication. However, we have checked that PubMed publishes all otorhinolaryngology journals listed in Journal Citation Reports (JCR), which reduced the probability of omitted publications in our study. Second, the median time to publication in our study transcended the provided 18 months after trial completion to publish the results, meaning that part of the registered trials completed in 2015 might not have been published yet. As a consequence, this could have underestimated the rate of publication and overestimated the probability of publication bias. Finally, investigator bias may have been introduced. The second investigator (IvB) was not blinded for the outcome of the standardized search for publications, conducted by the first investigator (JvH). This may have resolved, subconsciously, in gathering data selectively. However, the risk of bias was reduced by performing the search independently and by a third investigator who reviewed the discrepant data.
Conclusions
In summary, we showed that otology trails are prone to publication bias. Trials with a biological intervention were more likely to be published than studies involving drugs, and trials funded by industry were less likely to be published. Otology trials are published at a considerably low rate, and time to publication is still extensive. With little citations on ClinicalTrials.gov and low reporting of the National Clinical Trial registry number, the accessibility is limited. Availability of results and traceability of publications and corresponding trials are a clinical and ethical necessity. The benefits of health research can only be realized when study methods and results are submitted or published timely and accurately. Efforts must be made to raise awareness of publication bias and to further enhance accuracy of clinical trial registries, in order to diminish selective publication and non-publication.
Supporting information
(XLSX)
(DOCX)
Data Availability
All relevant data are within the manuscript and its Supporting Information files.
Funding Statement
The authors received no specific funding for this work.
References
- 1.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009. July;374(9683): 86–9. 10.1016/S0140-6736(09)60329-9 [DOI] [PubMed] [Google Scholar]
- 2.Cochrane Methods Bias—Reporting biases [Internet]. 2018 [cited 2018 Feb 12]. Available from: http://methods.cochrane.org/bias/reporting-biases
- 3.Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis. Sim I, editor. PLoS Med. 2009. September 8;6(9): e1000144 10.1371/journal.pmed.1000144 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014. January;383(9913): 257–66. 10.1016/S0140-6736(13)62296-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Khan NA, Singh M, Spencer HJ, Torralba KD. Randomized Controlled Trials of Rheumatoid Arthritis Registered at ClinicalTrials.gov: What Gets Published and When. Arthritis Rheumatol. 2014. October;66(10): 2664–74. 10.1002/art.38784 [DOI] [PubMed] [Google Scholar]
- 6.Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: Cross sectional analysis. BMJ. 2012;344(7838): 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153(3): 158–66. 10.7326/0003-4819-153-3-201008030-00006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Prenner JL, Driscoll SJ, Fine HF, Salz DA, Roth DB. Publication rates of registered clinical trials in macular degeneration. Retina. 2011;31(2): 401–4. 10.1097/IAE.0b013e3181eef2ad [DOI] [PubMed] [Google Scholar]
- 9.Wildt S, Krag A, Gluud L. Characteristics of randomised trials on diseases in the digestive system registered in ClinicalTrials.gov: A retrospective analysis. BMJ Open. 2011;1(2): 1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Shamliyan T, Kane RL. Clinical Research Involving Children: Registration, Completeness, and Publication. Pediatrics. 2012. May 1;129(5): e1291–300. 10.1542/peds.2010-2847 [DOI] [PubMed] [Google Scholar]
- 11.Smith HN, Bhandari M, Mahomed NN, Jan M, Gandhi R. Comparison of arthroplasty trial publications after registration in clinicaltrials.gov. J Arthroplasty. 2012;27(7): 1283–8. 10.1016/j.arth.2011.11.005 [DOI] [PubMed] [Google Scholar]
- 12.Hurley MN, Prayle AP, Smyth AR. Delayed publication of clinical trials in cystic fibrosis. J Cyst Fibros. 2012. January;11(1): 14–7. 10.1016/j.jcf.2011.08.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A. Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ. 2014. December 9;349(dec09 1): g6870–g6870. 10.1136/bmj.g6870 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chan AW, Pello A, Kitchen J, Axentiev A, Virtanen JI, Liu A, et al. Association of trial registration with reporting of primary outcomes in protocols and publications. JAMA—J Am Med Assoc. 2017;318(17): 1709–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009. January 21;(1): 5–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hopewell S, Clarke MJ, Stewart L, Tierney J. Time to publication for results of clinical trials. Cochrane Database Syst Rev. 2007. April 18;(2): 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004. September 16;351(12): 1250–1. 10.1056/NEJMe048225 [DOI] [PubMed] [Google Scholar]
- 18.Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with Results Reporting at ClinicalTrials.gov. N Engl J Med. 2015. March 12;372(11): 1031–9. 10.1056/NEJMsa1409364 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gill CJ. How often do US-based human subjects research studies register on time, and how often do they post their results? A statistical analysis of the Clinicaltrials.gov database. BMJ Open. 2012. August 29;2(4): e001186 10.1136/bmjopen-2012-001186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Law MR, Kawasumi Y, Morgan SG. Despite Law, Fewer Than One In Eight Completed Studies Of Drugs And Biologics Are Reported On Time On ClinicalTrials.gov. Health Aff. 2011. December 1;30(12): 2338–45. [DOI] [PubMed] [Google Scholar]
- 21.Grants.nih.gov [Internet]. NIH Policy on the Dissemination of NIH-Funded Clinical Trial Information. 2017 [cited 2019 Feb 27]. Available from: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-149.html
- 22.Regulation (EU) No. 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC. Official Journal of the European Union. 2014;L158: 5,29,35. [Google Scholar]
- 23.Ec.europa.eu [Internet]. Clinical trials—Regulation EU No 536/2014. 2018 [cited 2019 Feb 27]. Available from: https://ec.europa.eu/health/human-use/clinical-trials/regulation_nl
- 24.World Medical Association. World Medical Association Declaration of Helsinki. Bull world Heal Organ. 2001;79(4): 373–4. [PMC free article] [PubMed] [Google Scholar]
- 25.Prsinfo.clinicaltrials.gov [Internet]. ClinicalTrials.gov Protocol Registration Data Element Definitions for Interventional and Observational Studies. 2018 [cited 2018 Feb 6]. Available from: https://prsinfo.clinicaltrials.gov/definitions.html#PrimaryCompletionDate
- 26.Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012. January 3;344(jan03 1): d7373–d7373. 10.1136/bmj.d7373 [DOI] [PubMed] [Google Scholar]
- 27.Dickersin K, Rennie D. The evolution of trials registries and their use to assess the clinical trial enterprise. Jama. 2012;307(17)(17): 1861–4. 10.1001/jama.2012.4230 [DOI] [PubMed] [Google Scholar]
- 28.Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ. 1997. September 13;315(7109): 640–5. 10.1136/bmj.315.7109.640 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Chahal J, Tomescu SS, Ravi B, Bach BR, Ogilvie-Harris D, Mohamed NN, et al. Publication of sports medicine-related randomized controlled trials registered in clinicaltrials.gov. Am J Sports Med. 2012;40(9): 1970–7. 10.1177/0363546512448363 [DOI] [PubMed] [Google Scholar]
- 30.Fleminger J, Goldacre B. Prevalence of clinical trial status discrepancies: A cross-sectional study of 10,492 trials registered on both ClinicalTrials.gov and the European Union Clinical Trials Register. Beiki O, editor. PLoS One. 2018. March 7;13(3): e0193088 10.1371/journal.pone.0193088 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Zarin DA, Keselman A. Registering a Clinical Trial in ClinicalTrials.gov. Chest. 2007;131(3): 909–12 10.1378/chest.06-2450 [DOI] [PubMed] [Google Scholar]
- 32.Richter TA. Clinical Research: A Globalized Network. PLoS ONE. 2014;9(12): 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Huser V, Cimino JJ. Evaluating adherence to the International Committee of Medical Journal Editors' policy of mandatory, timely clinical trial registration. J Am Med Inform Assoc. 2013;20(1): 169–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(XLSX)
(DOCX)
Data Availability Statement
All relevant data are within the manuscript and its Supporting Information files.