Abstract
This study characterizes discrepancies between primary outcomes specified in trial protocols, registrations, and publications, and the association between prospective registration and consistency of published outcomes.
A major aim of trial registration is to help identify and deter the selective reporting of outcomes based on the results. However, it is unclear whether registered outcomes accurately reflect the trial protocol and whether registration improves the reporting of primary outcomes in publications. We evaluated adherence to trial registration and its association with subsequent publication and reporting of primary outcomes.
Methods
We conducted a cohort study of all initiated clinical trial protocols approved in 2007 by the research ethics committee for the region of Helsinki and Uusimaa, Finland. Registry records and articles published up to February 2017 were identified using keywords to search trial registries, PubMed, EMBASE, Cochrane Central, Finnish databases (Medic, ARTO, TUHAT), and Google. Trial characteristics and outcomes were extracted in duplicate from each protocol (including amendments), registry record, and publication.
Using descriptive statistics and multivariable logistic regression adjusting for characteristics in Table 1, we determined the prevalence of and variables associated with prospective registration (within 1 month after the trial start date to allow for incomplete start dates and processing delays in the registry); the proportion of trials with at least 1 discrepant primary outcome in the protocol compared with (1) the registry and (2) the publication; and the association between prospective registration and subsequent publication without discrepant primary outcomes compared with the protocol. A 2-sided P value of less than .05 was used for statistical significance, and odds ratios (ORs) with 95% CIs were calculated using Stata/SE (StataCorp), version 12.1.
Table 1. Study Characteristics Associated With Prospective Registration, Publication, and Publication Without Discrepant Primary Outcomes.
| Trial Characteristic | Clinical Trials, No. (N = 113) |
Registered | Publisheda | Published Without Discrepant Primary Outcomesb | |||
|---|---|---|---|---|---|---|---|
| No. (%)c | AOR (95% CI)d | No. (%)c | AOR (95% CI)d | No. (%)c | AOR (95% CI)d | ||
| Prospectively registered | |||||||
| Yes | 69 | 47 (68) | 4.53 (1.12-18.34) | 44 (64) | 5.79 (1.42-23.65) | ||
| No | 44 | 17 (39) | 1 [Reference] | 11 (25) | 1 [Reference] | ||
| Intervention type | |||||||
| Drug or biologic | 75 | 64 (85) | 30.99 (7.70-124.66) | 45 (60) | 0.30 (0.07-1.29) | 42 (56) | 0.46 (0.11-2.01) |
| Other | 38 | 5 (13) | 1 [Reference] | 19 (50) | 1 [Reference] | 13 (34) | 1 [Reference] |
| Planned sample sizee | |||||||
| ≥200 | 62 | 48 (77) | 4.41 (1.18-16.45) | 45 (73) | 2.75 (1.09-6.92) | 40 (65) | 2.69 (1.06-6.83) |
| <200 | 51 | 21 (41) | 1 [Reference] | 19 (37) | 1 [Reference] | 15 (29) | 1 [Reference] |
| No. of sites | |||||||
| Multicenter | 88 | 62 (70) | 1.82 (0.46-7.22) | 56 (64) | 1.80 (0.58-5.58) | 48 (55) | 1.09 (0.33-3.57) |
| Single center | 25 | 7 (28) | 1 [Reference] | 8 (32) | 1 [Reference] | 7 (28) | 1 [Reference] |
| Sponsor | |||||||
| Industry | 53 | 46 (87) | 1.97 (0.50-7.81) | 36 (68) | 1.23 (0.42-3.61) | 33 (62) | 1.35 (0.47-3.89) |
| Non-industry | 60 | 23 (38) | 1 [Reference] | 28 (47) | 1 [Reference] | 22 (37) | 1 [Reference] |
| Design | |||||||
| Controlled | 99 | 61 (62) | 0.95 (0.16-5.79) | 56 (57) | 1.09 (0.30-3.88) | 49 (49) | 1.39 (0.39-4.98) |
| Uncontrolled | 14 | 8 (57) | 1 [Reference] | 8 (57) | 1 [Reference] | 6 (43) | 1 [Reference] |
Abbreviation: AOR, adjusted odds ratio.
Publication year ranged from 2008 to 2016; median time from ethics approval to publication was 5 y (interquartile range, 3.5-6).
Published article compared with protocol.
Row percentage.
Multivariable logistic regression adjusting for listed trial characteristics.
Median sample size was 200 (interquartile range, 70-732).
Discrepancies were defined as (1) a new primary outcome being reported that was not specified as primary in the protocol; or (2) a protocol-defined primary outcome being omitted or downgraded (reported as secondary or unspecified) in the registry or publication. For comparison with registries, we used the primary outcomes defined in the most recent protocol or amendment dated before the initial registration date. For comparison with publications, we used the most recent protocol version regardless of amendment date.
Results
Among 113 trials, 69 (61%) were prospectively registered and 64 (57%) were published. Trials involving drug or biologic interventions and larger sample sizes were more likely to be registered (Table 1).
A primary outcome was not defined in 23 protocols (20%). Discrepancies were found in at least 1 primary outcome defined in the registry for 16 of 69 prospectively registered trials (23%) when compared with the protocol, whereas 9 of 58 published trials (16%) with defined primary outcomes had discrepancies between the publication and the protocol (Table 2). Discrepancies between the protocol and publication were more common in unregistered trials (6 of 11 trials [55%]) than registered trials (3 of 47 [6%]) (P < .001). Only 1 published article acknowledged the changes to primary outcomes.
Table 2. Proportion of Trials With Discrepancies in Primary Outcomes When Comparing Protocols With Prospective Registry Records and Published Articles.
| Discrepancya | No. of Trials With Discrepancies for ≥1 Primary Outcome/ Total Trials (%) | |
|---|---|---|
| Registry vs Protocol | Published Article vs Protocol | |
| Changed protocol-defined primary outcome | 13/67 (19)b | 5/55 (9)e |
| Reported as nonprimary | 2/67 (3)b | 5/55 (9)e |
| Omitted | 12/67 (18)b | 1/55 (2)e |
| New primary outcome | 3/63 (5)c | 7/56 (13)f |
| Changed from nonprimary to primary | 3/63 (5)c | 6/56 (11)f |
| Not listed in protocol | 1/63 (2)c | 1/56 (2)f |
| Any discrepancy in primary outcome | 16/69 (23)d | 9/58 (16)g |
Categories are not mutually exclusive; a trial could have more than 1 type of discrepancy for different primary outcomes.
Denominator represents registered trials that defined at least 1 primary outcome in the protocol.
Denominator represents registered trials that defined at least 1 primary outcome in the registry.
Denominator represents registered trials that defined at least 1 primary outcome in either the protocol or registry.
Denominator represents published trials that defined at least 1 primary outcome in the protocol.
Denominator represents published trials that defined at least 1 primary outcome in the published article.
Denominator represents published trials that defined at least 1 primary outcome in either the protocol or published article. Five trials had only discrepancies that were favorable to the main intervention, 3 had only unfavorable discrepancies, and 1 had a neutral combination of both favorable and unfavorable discrepancies.
Prospective registration was significantly associated with subsequent publication (68% of registered trials vs 39% of unregistered trials; adjusted OR, 4.53 [95% CI, 1.12-18.34]) (Table 1). Registered trials were also significantly more likely than unregistered trials to be subsequently published with the same primary outcomes as defined in the protocol (64% of registered trials vs 25% of unregistered trials; adjusted OR, 5.79 [95% CI, 1.42-23.65]).
Discussion
Clinical trials were often unregistered, unpublished, and discrepant in the reporting of primary outcomes across information sources. Limitations include the unclear generalizability beyond the Finnish jurisdiction and the limited sample size.
Although discrepancies are commonly found between registries and publications, which may reflect selective outcome reporting, the rationale is less clear for different primary outcomes appearing between the registry and protocol prior to results being known. Potential reasons for such discrepancies include clerical oversight or intentional suppression from disclosure. The original protocol and amendments should be made publicly available so that editors, peer reviewers, and readers can identify any unacknowledged changes to protocol-defined outcomes in the registry or publication. The protocol should provide a complete description of the primary outcomes and other key elements of the study plans. Amendments should be transparently reported.
Prospective registration was associated with publication and publication without discrepancies in the primary outcomes. Journal editors, regulators, research ethics committees, funders, and sponsors should implement policies mandating prospective registration for all clinical trials. Only with accessible, complete information can interventions be adequately evaluated for patient care.
Section Editor: Jody W. Zylke, MD, Deputy Editor.
References
- 1.Dwan K, Gamble C, Williamson PR, Kirkham JJ; Reporting Bias Group . Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS One. 2013;8(7):e66844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457-2465. [DOI] [PubMed] [Google Scholar]
- 3.Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13:282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Zarin DA, Tse T, Williams RJ, Carr S. Trial reporting in ClinicalTrials.gov—the final rule. N Engl J Med. 2016;375(20):1998-2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chan A-W, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257-266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200-207. [DOI] [PMC free article] [PubMed] [Google Scholar]