Skip to main content
NCBI home page
The BMJ logoLink to The BMJ
. 2018 Sep 12;362:k3218. doi: 10.1136/bmj.k3218

Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource

Ben Goldacre 1,✉, Nicholas J DeVito 1, Carl Heneghan 2, Francis Irving 1, Seb Bacon 1, Jessica Fleminger 1, Helen Curtis 1
PMCID: PMC6134801  PMID: 30209058

Abstract

Objectives

To ascertain compliance rates with the European Commission’s requirement that all trials on the EU Clinical Trials Register (EUCTR) post results to the registry within 12 months of completion (final compliance date 21 December 2016); to identify features associated with non-compliance; to rank sponsors by compliance; and to build a tool for live ongoing audit of compliance.

Design

Retrospective cohort study.

Setting

EUCTR.

Participants

7274 of 11 531 trials listed as completed on EUCTR and where results could be established as due.

Main outcome measure

Publication of results on EUCTR.

Results

Of 7274 trials where results were due, 49.5% (95% confidence interval 48.4% to 50.7%) reported results. Trials with a commercial sponsor were substantially more likely to post results than those with a non-commercial sponsor (68.1% v 11.0%, adjusted odds ratio 23.2, 95% confidence interval 19.2 to 28.2); as were trials by a sponsor who conducted a large number of trials (77.9% v 18.4%, adjusted odds ratio 18.4, 15.3 to 22.1). More recent trials were more likely to report results (per year odds ratio 1.05, 95% confidence interval 1.03 to 1.07). Extensive evidence was found of errors, omissions, and contradictory entries in EUCTR data that prevented ascertainment of compliance for some trials.

Conclusions

Compliance with the European Commission requirement for all trials to post results on to the EUCTR within 12 months of completion has been poor, with half of all trials non-compliant. EU registry data commonly contain inconsistencies that might prevent even regulators assessing compliance. Accessible and timely information on the compliance status of each individual trial and sponsor may help to improve reporting rates.

Introduction

The results of clinical trials are used by clinicians, patients, and policy makers to make informed choices about the benefits and safety of interventions. Sharing the methods and results of all trials has therefore long been recognised as an ethical and scientific imperative.1 2 3 More recently, institutions such as the World Health Organization, European Commission, and US Food and Drug Administration have called for the disclosure of results.4 5 6 However, there is extensive and longstanding evidence that the methods and results of completed clinical trials are commonly left unreported. A 2014 systematic review identified 22 cohorts of studies included in trial registries: half were not published in a journal (54.2%, 95% confidence interval 42.0% to 65.9%), 17 that were following up cohorts of trials approved by ethics committees yielded similar rates of unpublished results, and studies with statistically significant results were more likely to be published (odds ratio 2.8, 95% confidence interval 2.2 to 3.5).7 These findings are consistent with a previous review.8

In the US, the FDA Amendments Act 2007 (FDAAA) requires sponsors to post results on to ClinicalTrials.gov itself, rather than be published in a journal, within 12 months of completion for certain categories of trial.6 The two cohort studies published to date on this topic report compliance rates of only one trial in five.9 10 FDAAA, however, has complexities and limitations. Importantly, not all trials on ClinicalTrials.gov are covered by the requirement to report results on to the register—only those meeting certain criteria; and there is no data field on ClinicalTrials.gov to easily identify the subset of trials required to report results. Furthermore, although a list of trials with certificates of exemption from reporting can be obtained and used as a proxy to help identify those trials not covered by the requirement to report results, in practice many sponsors have only requested these certificates when seeking prospective clarity from the regulator on individual trials where exemption may be contentious; therefore, many exempt trials have no such certification. In addition, the “final rule” that gives further detail on which trials are covered and sets out the process for addressing breaches, was not published until 2016; and the first few trials to be legally covered by this rule have only recently become due to report results.11 In addition, the final rule changed the number of trials covered by the act: trials on unapproved products completing before January 2017 are no longer required to post results after the product is approved.

In comparison, the European Commission is moving towards more straightforward transparency rules. Any trial of any medicinal product conducted since 2004 in an EU country has already been required to register on the European Union Clinical Trials Register (EUCTR), which is administered by the European Medicines Agency (EMA). Following the 2012 EC guideline 2012/c302/03, sponsors must ensure that all trials registered on EUCTR since 2004 disclose their results to the EMA within 12 months of trial completion; phase I trials are exempt unless they are denoted as being part of a paediatric investigation plan.12 These trial reports are posted publicly on to the EUCTR within 15 working days of receipt by the EMA and are required to include salient features such as results for all prespecified trial outcomes and statistical analyses, details of “serious” and “non-serious” adverse events, participants’ baseline characteristics, and protocol deviations, as well as discussion of design limitations and caveats.13 Following various delays in the EMA implementing the software platform for results posting, the final date for sponsors’ compliance was 21 December 2016.14 15

We assessed compliance with the EU requirement to post results on to EUCTR for all trials on the registry, explored factors associated with non-compliance, identified the individual trial sponsors that are best at complying, and created a live online service, driven by regular updates of the EUCTR data, to give ongoing and regularly updated performance statistics for compliance.

Methods

Data sources

We downloaded trial records for all trials from the EUCTR database in the week commencing 17 January 2018 using bespoke software produced for OpenTrials.net,16 an open database of publicly accessible documents and data on clinical trials. Fifteen trials were randomly selected, and to ensure the download was correctly accessing data we manually checked variables in the downloaded data against the EUCTR website.

The structure of the data on EUCTR is different to that of other registries: each trial can be conducted in multiple countries, and within each trial, each country where the trial is being conducted has a separate register entry; these are linked by a single common trial identity number, with a country code as suffix. Basic information such as trial phase, completion date, and completion status can be discrepant between countries’ entries for the same trial: while discrepancies for some data fields (such as phase) may be errors, others may reflect true differences in the conduct of a trial between countries. For the analysis we collapsed the protocol data for individual countries into a single entry for each trial based on the unique trial identifier (EudraCT Number). We extracted basic information on each trial, including trial identity number, sponsor name, and sponsor class (commercial, non-commercial). We then generated variables for several features of each trial: earliest country “global end of trial date”, latest country global end of trial date, and trial status (“ongoing”; all countries “complete” or “terminated,” some but not all countries “complete” or “terminated,” or other (suspended or no status given)). Based on the presence of a link to results in the results field in EUCTR, we also created a variable for whether a trial has reported results. No other metric of whether results have been reported on EUCTR exists, and all trials that have reported have a link in this field.

Inclusion and exclusion criteria

Our study population was all trials where results were due under the 2012 guideline. Specifically, this included all trials where all countries’ register entries are marked as complete or terminated. We excluded trials where the latest country completion date was more recent than 19 December 2016 to allow 12 months for results reporting, as per the official trial reporting requirements, and 15 additional working days for EMA to publish submitted results. We also excluded trials marked as completed in all countries but where no global date of the end of the trial was given in any country’s record; although these trials should have a completion date, they do not, and so it cannot be ascertained whether their results are due. We also excluded all phase I trials unless they were part of a paediatric investigation plan.

Explanatory variables

We created variables for a range of features of each trial, selected prospectively on the basis of clinical and methodological interest. If there were discrepancies between country protocols for the same trial on any data element, then we coded that trial as discordant between countries for the variable in question. The following variables were generated: phase (I, II, III, IV, or discordant between countries), whether any country has noted the trial as being part of a paediatric investigation plan, whether the condition being studied is designated as a rare disease, whether the trial is a bioequivalence study, whether the participants were healthy volunteers, whether the trial was terminated (specifically, where all countries records were marked as terminated), whether a trial had multiple sponsors, whether a trial was conducted in multiple countries, and whether the sponsor name was missing or unclear. We attempted to generate variables on other features such as blinding, however structured data on these features was spread between multiple fields, which were often incomplete or inconsistent between countries.

Sponsor names are entered into the EUCTR as free text, and how the same sponsor is identified often varies, such as GSK Ltd and GlaxoSmithKline Limited, or Medical University Vienna and Medizinische Universität Wien. We therefore manually normalised the data in these fields, merging records under a single name. Separately, where possible, we also created an additional sponsor name variable that accounted for acquisitions and mergers among large companies as well as university hospital systems where warranted. We generated a variable containing the number of trials the sponsor of each trial sponsored, and divided this into quarters. The top fourth of this variable therefore contains trials sponsored by organisations that sponsor a large number of trials, such as large pharmaceutical companies, whereas the bottom fourth contains trials sponsored by those who sponsor very few trials, or only one trial ever.

Analysis

We generated descriptive statistics on both the characteristics of trials in the total EUCTR cohort and the study cohort of trials where results were due. We calculated the percentage of trials reported overall, and broken down by completion year, phase, and sponsor class, whether the trial was part of a paediatric investigation plan, whether the condition studied was designated on the register as a rare disease, and whether the trial was a bioequivalence study or conducted in healthy volunteers. The exact method was used to calculate confidence intervals. We constructed a logistic regression model with all these explanatory variables, as they were selected prospectively on the basis of clinical and methodological interest. The binary response variable in our logistic regression analysis was the presence of results for the trial. Lastly, we produced ranked lists of the sponsors with the largest number of reported and unreported trials.

Live data web resource

The EUCTR data underlying this study are updated on a regular basis. We therefore commissioned a software engineer (FI) to develop an interactive online website presenting the overall reporting rate for all due trials, the reporting rates for each sponsor, ranks for these reporting rates, and details of each sponsor’s individual reported and unreported trials. The data underlying this site update regularly following each new download of the EUCTR database: the results and ranks for each individual sponsor are therefore maintained as current, and they change as performance changes. All software underlying this service is shared as open source and available for open code review or for adaptation and re-use.17

Software, data sharing, and reproducibility

All underlying software is freely available online under an open source licence for review and re-use.17 Data were extracted from the database using SQL in PostgreSQL; statistical analyses were conducted using Stata 14. The full downloaded dataset and all analytical code are shared through Github.18

Patient involvement

The development of the overall research question and outcome measures was informed by the AllTrials campaign’s extensive engagement with signatories and supporters, including patient groups. Patients were not formally involved in developing the study design.

Results

Characteristics of included trials

The EUCTR database contained 31 821 trials in total. Overall, 11 345 trials (35.7%) were conducted in more than one country (mean number of countries per trial 2.5, median 1). Three trials were excluded as their final reported completion dates were given incorrectly on EUCTR (completion years 2019 and 2041, both in the future, even though this field is supposed to be retrospectively populated with the actual completion date of the trial; and 2000, before the registry began recording trials), leaving 31 818 trials. We excluded 20 287 trials as their status was not reported as either completed or terminated in all countries. Of the 11 531 trials remaining that were listed as completed or terminated in every country, 3392 (29.4%) were excluded because they had no completion date in any countries’ record. We excluded 540 trials because their completion date was within the past 12 months and results were therefore not yet due, and 325 trials were excluded as they were phase I and not part of a paediatric investigation plan. The final cohort of studies with results due therefore comprised 7274 trials. Figure 1 shows a flow diagram for all trials on EUCTR.

Fig 1.

Fig 1

Open in a new tab

Flow diagram for all trials on the EU Clinical Trials Register (EUCTR). PIP=paediatric investigation plan

Table 1 shows the characteristics of trials in the full EUCTR database and trials in the final cohort with results due. Trial completion dates were evenly spread over the preceding seven years, with fewer trials completing before 2008. Most trials with results due had a commercial sponsor (66.5%). Phase II trials were the most common (44.6%), followed by phase III (32.2%) and phase IV (21.7%). There were few trials on rare diseases (8.0%), bioequivalence (0.1%), or healthy volunteers (9.7%).

Table 1.

Characteristics of included trials

Variables No (%)
Total cohort (31 818 trials) Cohort with results due (7274 trials)
Results available 9876 (31.0) 3601 (49.5)
Completion year:
 2004 4 (0.0) 3 (0.0)
 2005 100 (0.3) 72 (1.0)
 2006 475 (1.5) 276 (3.8)
 2007 762 (2.4) 444 (6.1)
 2008 1121 (3.5) 658 (9.1)
 2009 1123 (3.5) 654 (9.0)
 2010 1143 (3.6) 699 (9.6)
 2011 1204 (3.8) 731 (10.1)
 2012 1137 (3.6) 713 (9.8)
 2013 1258 (4.0) 737 (10.1)
 2014 1251 (3.9) 803 (11.0)
 2015 1210 (3.8) 782 (10.8)
 2016 1180 (3.7) 702 (9.7)
 2017 1015 (3.2) -
 2018 4 (0.0) -
 Missing 18 831 (59.2) -
Trial status:
 All countries, ongoing 12 947 (40.7) -
 All countries, complete or terminated 11 531 (36.2) 7274 (100.0)
 Any countries, complete or terminated 5754 (18.1) -
 Other (eg, suspended) 519 (1.6) -
 Blank 1067 (3.4) -
Phase:
 Discordant between countries 398 (1.3) 83 (1.1)
 I 1569 (4.9) 23 (0.3)
 II 12 191 (38.3) 3247 (44.6)
 III 10 282 (32.3) 2340 (32.2)
 IV 7378 (23.2) 1581 (21.7)
Sponsor class:
 Non-commercial 14 408 (45.3) 2353 (32.4)
 Commercial 16 964 (53.3) 4837 (66.5)
 Mixed 265 (0.8) 57 (0.8)
 Blank 181 (0.6) 27 (0.4)
Part of paediatric investigation plan 1042 (3.3) 107 (1.5)
Condition being studied is rare disease:
 No 27 496 (86.4) 6584 (90.5)
 Yes 3723 (11.7) 584 (8.0)
 Discordant between countries 382 (1.2) 45 (0.6)
 Data not available 217 (0.7) 61 (0.8)
Bioequivalence study:
 No 31 738 (99.8) 7264 (99.9)
 Yes 66 (0.2) 9 (0.1)
 Discordant between countries 14 (0.0) 1 (0.0)
Participants are healthy volunteers:
 No 29 132 (91.6) 6555 (90.1)
 Yes 2638 (8.3) 708 (9.7)
 Discordant between countries 48 (0.2) 11 (0.2)
Total No of trials registered for trial’s sponsor:
 First quarter (1-10) 8146 (25.6) 1907 (26.2)
 Second quarter (11-53) 7971 (25.1) 1677 (23.1)
 Third quarter (54-244) 7953 (25.0) 1547 (21.3)
 Fourth quarter (274-1260) 7748 (24.4) 2143 (29.5)
No sponsor name given 145 (0.5) 18 (0.3)
Unclear sponsor name given 227 (0.7) 66 (0.9)
All sites terminated 2155 (6.8) 1062 (14.6)
No of countries:
 1 20 476 (64.4) 4536 (62.4)
 2 2611 (8.2) 941 (12.9)
 ≥3 8731 (27.4) 1797 (24.7)
Trial has multiple sponsors 982 (3.1) 145 (2.0)
Open in a new tab

Outcome data

In the main study cohort of 7274 trials where results were due, 3601 reported results (49.5%, 95% confidence interval 48.4% to 50.7%). Table 2 shows the proportion of trials reported overall at each level of each variable for the cohort of trials with results due. Results were reported for 68.1% of due trials with a commercial sponsor (95% confidence interval 66.7% to 69.4%) and 11.0% of trials with a non-commercial sponsor (9.8% to 12.4%). Trials conducted by sponsors with a large number of trials on the register had a higher proportion reported (18.4% (95% confidence interval 16.7% to 20.2%) for the lowest quarter, increasing to 77.9% (76.1% to 79.6%) for the highest quarter); appendix 1 contains a post hoc sensitivity analysis of reporting rates for this variable in smaller categories.

Table 2.

Reporting rates in each cohort, by category of trial

Variables Total trials Trials with results % with results (95% CI)
All due trials 7274 3601 49.5 (48.4 to 50.7)
Completion year:
 2004 3 1 33.3 (2.6 to 90.4)
 2005 72 44 61.1 (49.4 to 71.7)
 2006 276 154 55.8 (49.9 to 61.6)
 2007 444 241 54.3 (49.6 to 58.9)
 2008 658 326 49.5 (45.7 to 53.4)
 2009 654 323 49.4 (45.6 to 53.2)
 2010 699 340 48.6 (44.9 to 52.4)
 2011 731 366 50.1 (46.4 to 53.7)
 2012 713 347 48.7 (45.0 to 52.3)
 2013 737 353 47.9 (44.3 to 51.5)
 2014 803 378 47.1 (43.6 to 50.5)
 2015 782 410 52.4 (48.9 to 55.9)
 2016 702 318 45.3 (41.6 to 49.0)
Phase:
 Discordant 83 60 72.3 (61.7 to 80.9)
 I 23 20 87.0 (65.8 to 95.8)
 II 3247 1597 49.2 (47.5 to 50.9)
 III 2340 1421 60.7 (58.7 to 62.7)
 IV 1581 503 31.8 (29.6 to 34.2)
Sponsor class:
 Non-commercial 2353 260 11.0 (9.8 to 12.4)
 Commercial 4837 3292 68.1 (66.7 to 69.4)
 Mixed 57 45 78.9 (66.4 to 87.7)
 Blank 27 4 14.8 (5.6 to 33.9)
Paediatric investigation plan 107 81 75.7 (66.7 to 82.9)
Condition being studied is rare disease:
 No 6584 3280 49.8 (48.6 to 51.0)
 Yes 584 273 46.7 (42.7 to 50.8)
 Discordant 45 33 73.3 (58.5 to 84.3)
 Data unavailable 61 15 24.6 (15.3 to 37.0)
Bioequivalence study:
 No 7264 3595 49.5 (48.3 to 50.6)
 Yes 9 6 66.7 (31.5 to 89.7)
 Discordant 1 0 0.0
Participants are healthy volunteers:
 No 6555 3226 49.2 (48.0 to 50.4)
 Yes 708 364 51.4 (47.7 to 55.1)
 Discordant 11 11 100.0
Total No of trials registered for trial’s sponsor:
 First quarter (1-10) 1907 351 18.4 (16.7 to 20.2)
 Second quarter (11-53) 1677 656 39.1 (36.8 to 41.5)
 Third quarter (54-244) 1547 924 59.7 (57.3 to 62.1)
 Fourth quarter (274-1260) 2143 1670 77.9 (76.1 to 79.6)
No sponsor name given 18 0 0.0
Unclear sponsor name given 66 2 3.0 (0.8 to 11.4)
All sites terminated 1062 328 30.9 (28.2 to 33.7)
No of countries:
 1 4536 1559 34.4 (33.0 to 35.8)
 2 941 616 65.5 (62.4 to 68.4)
 ≥3 1797 1426 79.4 (77.4 to 81.2)
Trial has multiple sponsors 145 76 52.4 (44.3 to 60.4)
Open in a new tab

Table 3 shows the crude univariable and adjusted multivariable odds ratios for features potentially associated with trial reporting. In the adjusted multivariable analysis, trials with a commercial sponsor were significantly more likely to post results (adjusted odds ratio 23.3, 95% confidence interval 19.2 to 28.2); as were trials by a sponsor who conducted a large number of trials (18.4, 15.3 to 22.1). We note that odds ratios are often high in studies where the outcome is common. In the crude data, trials completing longer ago were more likely to report results, whereas in the adjusted multivariable analysis this relation was reversed and more recent trials were more likely to report results (per year odds ratio 1.05, 95% confidence interval 1.03 to 1.07). Terminated trials were less likely to report results (odds ratio 0.55, 95% confidence interval 0.45 to 0.66). No statistically significant relation was found between the odds of reporting results and trial phase, use of healthy volunteers, rare disease status, giving no clear sponsor name, having multiple sponsors, or being a bioequivalence study. Appendix 1 contains sensitivity analyses treating “completion year” as a categorical rather than a continuous variable, treating “sponsor’s number of trials” as a continuous variable rather than quarters, and retaining only one randomly selected trial for each sponsor.

Table 3.

Crude and adjusted odds ratios for factors associated with trial reporting

Variables Crude odds ratio (95% CI) P value Adjusted odds ratio (95% CI) P value
Completion year:
 Increase of one year 0.98 (0.96 to 0.99) 0.005 1.05 (1.03 to 1.07) <0.001
Phase:
 Discordant 1.69 (1.03 to 2.74) 0.04 1.48 (0.75 to 2.89) 0.26
 I 4.31 (1.28 to 14.53) 0.02 2.26 (0.50 to 10.28) 0.29
 II 0.63 (0.56 to 0.70) <0.001 1.09 (0.93 to 1.26) 0.29
 III Reference
 IV 0.30 (0.26 to 0.34) <0.001 1.13 (0.92 to 1.38) 0.26
Sponsor class:
 Non-commercial Reference
 Commercial 17.17 (14.89 to 19.80) <0.001 23.25 (19.15 to 28.24) <0.001
 Mixed 30.19 (15.76 to 57.81) <0.001 15.52 (7.29 to 33.01) <0.001
 Blank 1.40 (0.48 to 4.08) 0.54 2.27 (0.70 to 7.36) 0.17
Part of paediatric investigation plan 3.23 (2.07 to 5.03) <0.001 1.09 (0.59 to 2.01) 0.79
Condition being studied is rare disease:
 No Reference
 Yes 0.88 (0.75 to 1.05) 0.15 0.96 (0.76 to 1.22) 0.74
 Discordant 2.77 (1.43 to 5.37) 0.003 0.78 (0.34 to 1.77) 0.55
 Data not available 0.33 (0.18 to 0.59) <0.001 0.62 (0.26 to 1.45) 0.27
Bioequivalence study 2.04 (0.51 to 8.17) 0.31 5.16 (0.89 to 29.82) 0.07
Participants are healthy volunteers 1.09 (0.93 to 1.28) 0.27 1.29 (1.03 to 1.62) 0.02
Total No of trials registered for trial’s sponsor:
 First quarter (1-10) Reference
 Second quarter (11-53) 2.84 (2.44 to 3.31) <0.001 5.99 (4.98 to 7.20) <0.001
 Third quarter (54-244) 6.57 (5.63 to 7.67) <0.001 19.17 (15.54 to 23.64) <0.001
 Fourth quarter (274-1260) 15.67 (13.43 to 18.29) <0.001 18.38 (15.31 to 22.06) <0.001
All sites terminated 0.40 (0.35 to 0.46) <0.001 0.55 (0.45 to 0.66) <0.001
Unclear sponsor name given 0.03 (0.01 to 0.13) <0.001 1.65 (0.36 to 7.47) 0.52
No of countries:
 1 Reference
 2 3.63 (3.13 to 4.21) <0.001 1.31 (1.08 to 1.60) 0.01
 ≥3 7.34 (6.45 to 8.36) <0.001 1.89 (1.58 to 2.26) <0.001
Trial has multiple sponsors 1.13 (0.81 to 1.56) 0.48 1.63 (1.03 to 2.58) 0.04
Open in a new tab

Errors, omissions, and inconsistencies in EUCTR data

We found that omissions and inconsistencies were common in EUCTR data, and so we present an additional analysis of these issues. While the date for global end of the trial is expected to be consistent across different country’s register entries for the same trial, 1890 trials (5.9% of the total EUCTR database) had discrepant such dates between countries: the median difference between the earliest and latest completion date among trials with a discrepancy was 48 days (interquartile range 13-133). Data for the completion date were commonly missing: in 11 531 trials every country was reported as “completed” or “terminated”—all these trials should also report a completion date; 3392 trials (29.4%) did not.

Trials with no completion date could not be included in our analysis as it could not be ascertained whether their results were due. We are, however, able to explore the possible impact of trials with missing completion dates on the overall results reporting rates. Among the 7786 eligible trials that were reported as completed in every country and correctly gave a completion date, 7024 (90.2%) completed more than 12 months ago and therefore had results due, and 722 of the 3270 (22.1%) otherwise eligible trials with missing completion dates, reported results. It is therefore likely that many of the trials with missing dates were due to report results, but failed to do so.

We found related inconsistencies for trials with discrepancies on trial status between countries. In the full EUCTR database only some (but not all) countries were marked as completed or terminated for 5754 trials and therefore could not be included in our cohort of trials with results due, as some trial sites may have been genuinely ongoing. However, 4846 of these trials (84.2%) had at least one global end of the trial date, which should only be available when all countries for that trial have completed; this strongly suggests that many of these trials have inconsistent data on the register. It is not possible to ascertain whether these trials have results due; however, 3643 of such trials (63.3%) had reported results.

Sponsor rankings

Tables 4 and 5 present ranked lists for major sponsors with the highest and lowest proportion of reported trials. Only sponsors with more than 50 trials in total on the register are included. For these tables we do not attribute trials to sponsors by company acquisition or merger, only by sponsor name (at our online audit tool, trials that may be attributable to a sponsor due to acquisition or merger are listed, but separately). As expected from the crude reporting rates and the results of the multivariable analysis, the sponsors with the highest proportion of trials reported are overwhelmingly pharmaceutical companies, whereas the sponsors with the lowest reporting rates are universities.

Table 4.

Sponsors with highest proportion of trials reported

Sponsor Total trials on EUCTR Due trials Due trials with results % reported
Gilead Sciences 213 31 31 100.0
Chiesi Farmaceutici 94 37 37 100.0
CSL Behring 72 25 25 100.0
Alcon 71 20 20 100.0
Genentech 63 18 18 100.0
Vertex Pharmaceuticals 62 19 19 100.0
Daiichi Sankyo 62 12 12 100.0
Almirall 53 37 37 100.0
Ferring Pharmaceuticals 53 19 19 100.0
Sanofi 573 111 110 99.1
Bayer 274 72 71 98.6
Johnson and Johnson 424 108 106 98.1
Novo Nordisk 202 52 51 98.1
Servier Laboratories 134 48 47 97.9
Novartis Vaccines 142 44 43 97.7
Abbvie 179 33 32 97.0
H Lundbeck 76 29 28 96.6
Astrazeneca 520 141 136 96.5
Otsuka 58 27 26 96.3
Amgen 244 51 49 96.1
Pfizer 744 168 161 95.8
Takeda 172 47 45 95.7
Astellas 137 23 22 95.7
Bristol-Myers Squibb 314 36 34 94.4
Eisai 113 13 12 92.3
Boehringer Ingelheim 340 90 83 92.2
Biogen 103 35 32 91.4
Merck 662 164 146 89.0
GlaxoSmithKline 1060 293 260 88.7
Ipsen 74 25 22 88.0
Merck 149 33 29 87.9
Novartis 1260 473 415 87.7
UCB 180 40 35 87.5
Celgene 107 8 7 87.5
Roche 596 115 100 87.0
Abbott 109 57 47 82.5
University of Dundee 69 61 50 82.0
Actelion Pharmaceuticals 82 14 11 78.6
University of Oxford 102 26 20 76.9
Shire 98 17 13 76.5
Galderma R&D 54 24 18 75.0
Teva 81 25 18 72.0
EORTC 88 14 10 71.4
Menarini Group 64 23 16 69.6
Allergan 115 46 30 65.2
Pierre Fabre 117 18 11 61.1
Baxter 61 28 16 57.1
University of Leeds 57 14 7 50.0
Open in a new tab

EUCTR=EU Clinical Trials Register.

Table 5.

Sponsors with highest proportion of trials unreported

Sponsor Total trials on EUCTR Due trials with results Due trials % reported
Hospitals of Paris 194 0 7 0.0
Karolinska Institutet 189 0 21 0.0
Radboud University 178 0 3 0.0
Charité-Universitätsmedizin Berlin 177 0 63 0.0
Erasmus University 161 0 3 0.0
University of Amsterdam 153 0 4 0.0
Agostino Gemelli University Polyclinic 142 0 11 0.0
Ghent University 126 0 19 0.0
VU University Medical Centre 126 0 3 0.0
Utrecht University 122 0 6 0.0
AOU di Bologna, Policlinico S.Orsola-Malpighi 120 0 1 0.0
Helsinki University 101 0 12 0.0
Université libre de Bruxelles 85 0 3 0.0
Vita-Salute San Raffaele University 83 0 5 0.0
Hospices Civils de Lyon 81 0 3 0.0
Heidelberg University 75 0 17 0.0
University of Oslo 72 0 1 0.0
University of Munich (Ludwig-Maximilians) 71 0 26 0.0
Maastricht University 61 0 2 0.0
Fundació Clínic per a la Recerca Biomèdica 60 0 1 0.0
University of Cologne 57 0 18 0.0
Gothenburg University 56 0 6 0.0
Uppsala University/Uppsala County Council 55 0 6 0.0
Manchester University NHS Foundation Trust 54 0 13 0.0
European Institute of Oncology 54 0 1 0.0
Blaise Pascal University 53 0 4 0.0
Hospital de la Santa Creu i Sant Pau 53 0 3 0.0
Odense University Hospital 88 1 27 3.7
Technical University of Munich 61 1 27 3.7
Medical University of Graz 105 2 53 3.8
Medical University of Vienna 354 8 166 4.8
University of Nottingham 58 1 17 5.9
Belfast Health and Social Care Trust 50 1 16 6.2
Copenhagen University and Hospitals 395 9 133 6.8
NHS Greater Glasgow and Clyde 52 1 13 7.7
KU Leuven 192 1 8 12.5
Guy's and St Thomas' NHS Foundation Trust 64 1 8 12.5
University of Birmingham 83 2 13 15.4
Aarhus University 134 7 41 17.1
King’s College London 90 2 11 18.2
Innsbruck Medical University 59 3 15 20.0
No sponsor name given 184 6 29 20.7
Imperial College London 118 5 19 26.3
Newcastle upon Tyne NHS Foundation Trust 59 4 14 28.6
The Royal Marsden NHS Foundation Trust 51 2 5 40.0
University College London 113 9 20 45.0
Eli Lilly 375 41 86 47.7
University of Leeds 57 7 14 50.0
Open in a new tab

EUCTR=EU Clinical Trials Register.

Live data web resource

The live data tool was successfully delivered. Figure 2 shows a screenshot of an arbitrarily selected sponsor’s page, showing sponsor’s summary results, followed by the list of their individual trials, and reporting status for each trial. The data on the site update every month, and all current data can be viewed online (EU.TrialsTracker.net). The names of any new sponsor are manually matched against the existing list of sponsors as appropriate on a monthly basis. Over time—for example, through company acquisition or merger—one current sponsor may become responsible for previous trials from another listed sponsor: this is reflected at the bottom of each sponsors page, where suggested additional sponsors are listed for review by users.

Fig 2.

Fig 2

Open in a new tab

Screenshot of single sponsor page on EU.TrialsTracker.net

Discussion

Compliance with the European Commission requirement for trial results to be reported to the EU Clinical Trials Register (EUCTR) is poor: only half (49.5%) of 7274 due trials have reported results. Trials with commercial sponsors were substantially more likely to post results, as were trials by a sponsor who conducted a large number of trials. Unexpectedly, we also found extensive evidence of omissions and contradictory data in EUCTR—notably, that 29.4% of trials marked as completed gave no completion date, which prevents ascertainment of compliance with reporting requirements. We also found evidence that completed trials were mislabelled on EUCTR as ongoing in some countries.

Strengths and weaknesses of this study

To our knowledge this is the first study of compliance with European Commission requirements on trials transparency, covering all trials of medicinal products conducted over a 12 year period in a territory of 500 million people, for the second largest trials registry in the world. Compliance rates with reporting requirements can be arguably ascertained more accurately for EUCTR than for ClinicalTrials.gov, as the inclusive nature of European provisions for transparency means that all trials on the EU register are required to report results within 12 months of completion; whereas the FDA Amendments Act 2007 (FDAAA) permits various exemptions from reporting requirements that cannot reliably be extracted automatically across a large volume of past trials from trial metadata on ClinicalTrials.gov.

The European Commission guideline requires the results of all trials to be reported in structured form on to the register itself. Ascertainment of the outcome—a results report on EUCTR—was therefore accurate and complete. It is possible that some trials that did not report results to EUCTR did report results elsewhere—for example, in a conference presentation, an academic journal article, as part of a meta-analysis after data were requested by systematic reviewers, or in the grey literature. Such publications do not meet the reporting requirements of the European Commission guideline and are therefore outside the scope of our study, as with previous studies on compliance with FDAAA requirements to report results on to ClinicalTrials.gov.9 10 We conducted a manual search of academic journals and grey literature for a random sample of 100 trials unreported on EUCTR, as requested during peer review for this paper (see appendix 2): 46 had results in a journal publication and five in the grey literature. Ascertainment of results publication by manual searches in academic journals and other sources for a complete cohort of trials on a register is time consuming, cannot be done with perfect accuracy, and cannot be repeated on a regular cycle of audit such as our online audit tool, where the data are updated on a monthly basis. This reflects an important advantage of rules requiring trial results to be reported directly on to a register rather than elsewhere. In addition, results reported in standardised formats to a registry may be more reliable than journal publication: both FDAAA and EU rules require complete reporting of prespecified outcomes, analyses, and adverse events; whereas reporting quality is highly variable in journal publications.19 Furthermore, two large cohort studies of 202 and 110 trials have now reported that structured results reports posted on ClinicalTrials.gov present more complete data on both results and adverse events than do traditional journal publications.20 21

We were able to identify the large cohort of trials where results were definitely due; however, omissions and inconsistencies in EUCTR data presented challenges for assessing compliance with reporting requirements for an additional subset of trials, where it was only possible to ascertain that the data on the register were flawed: 29.4% of trials listed as entirely completed in EUCTR gave no completion date, even though one is required, which made it impossible to assess whether results were due for these trials. Overall, reporting rates were worse in this subset of trials than in the cohort with consistent data; we may therefore have over-estimated compliance.

Findings in context

To our knowledge this is the first study of compliance with European Commission requirements on trial reporting. Our findings are consistent with those summarised in the most current systematic review on publication rates from 2014, which included 39 cohorts and found journal publication rates of 46.2% (95% confidence interval 40.2% to 52.4%) for trials approved by ethics committees and 54.2% (42.0% to 65.9%) for trials on trial registers; and with a previous review from 2010.7 8 Two cohort studies in 2012 and 2015 have assessed compliance with FDAAA and found compliance rates of only one trial in five.9 10 However, these estimates may be inaccurate owing to the challenges in ascertaining whether trials are exempt from FDAAA reporting requirements; and they relate to compliance with FDAAA 2007 before changes made in the 2016 Final Rule on the Act. Various further studies have assessed reporting rates on ClinicalTrials.gov for a cohort of studies without formally assessing compliance with the 12 month reporting requirements of FDAAA, and found similar proportions reported.22 23 24

Policy implications

We have found strong evidence that the European Commission guideline, requiring all trials’ results to be reported on EUCTR within 12 months of completion, is commonly being breached. Sponsors doing fewer trials, and non-commercial sponsors such as universities, have particularly low reporting rates: they may be more likely to be unaware of their obligations or lack administrative procedures to flag breaches and support compliance among their researchers. They may also lack clear lines of responsibility: in law, the sponsor is responsible for reporting the trial on to the register; in reality, it falls to the principal investigator or administrative staff. This may be particularly problematic for the smaller cohort of trials newly required (since 2016) to report results on to the register, but that completed in 2005 and have been left unreported; as staff may have moved jobs or even retired. We encourage EU universities to prioritise clarifying these issues for their staff and investing in basic internal audits and administrative work to ensure that results are reported on time. It is possible that enforcement notices or penalties would improve compliance and raise awareness of the obligation to report all trial results: these may become commonplace with the enactment of the new 2014 European Clinical Trials Regulation, which will come into force by 2022. In the absence of formal legal sanctions, public accountability and audit have valuable roles. The presence of a public ranking of sponsors’ reporting performance may encourage organisations to prioritise results reporting in general. In addition, the online resource we have produced also makes it easy for sponsors to identify individual trials from their organisations which have not yet reported results to EUCTR; it therefore offers practical support for sponsors wanting to improve compliance.

Although poor reporting rates in some sectors is a source of concern, the extremely high rate of compliance among commercial sponsors conducting a large number of trials is positive: it shows that, with an unambiguous requirement for all trials to report results, near perfect compliance can practically be delivered. In addition, since transparency requirements are relatively new, compliance may improve over time: we will assess this in future research and through routine monthly updates on the accompanying website (EU.TrialsTracker.net). We are concerned by extensive omissions and contradictory data in trial register entries on EUCTR. In some cases these errors were critical and made it impossible to ascertain the compliance status of a trial. If EUCTR is the only source of data to regulators, then it does not contain the information needed for them to establish whether all trials are compliant with European Commission guidelines on transparency. While sponsors are responsible for entering correct data, omissions and inconsistencies could be monitored and addressed by the European Medicines Agency, by running the same checks on its EUCTR database that we have run for this analysis.

Future research

Typically, publication bias research is retrospective—published long after a cohort of trials have completed—and presents only a single static estimate of overall performance for a population of trials. Static retrospective analyses such as these may not be the most effective use of analytic resources on registry and reporting data, which could be an important source of feedback to improve reporting in individual organisations. Quality improvement work through audit typically aims to identify good performers, learn from their successes, and help those with poor performance to improve. To be effective, audit should give timely, relevant, and actionable data, be repeated, and ideally be ongoing.25 These principles can be readily applied to clinical trials reporting, as we have done in this paper and the associated live data tool online (EU.TrialsTracker.net). From the launch of the associated online tool, using feedback from end users such as policy makers and the research community, we aim to learn how best to implement live feedback on reporting rates and information on individual unreported trials, for maximum usability and positive impact.

Conclusions

Compliance with the European Commission guideline, which aims to ensure that all trials report results within 12 months of completion, has been poor. Half of all due trials have not yet reported results. However, sponsors conducting a large number of trials, and pharmaceutical companies, show higher rates of compliance. We hope that accessible and timely information on the compliance status of each individual trial and sponsor will help to improve reporting rates.

What is already known on this topic

  • Numerous cohort studies have shown that the results of clinical trials are routinely left unreported

  • 2007 legislation that was intended to fix this problem in the US has been widely ignored

  • Recent EU rules require all trials conducted in Europe on medicinal products to report results directly on to the EU Clinical Trials Register (EUCTR) within 12 months of trial completion; but compliance has never been assessed

What this study adds

  • Compliance with the EU rules has been poor overall, compliance among pharmaceutical companies has been good, and universities have performed poorly

  • A live online searchable web resource was created, showing the reporting status of every individual trial conducted in Europe and overall performance rankings for every sponsor

  • This openly accessible data is updated every month

Acknowledgments

We thank Open Knowledge International for work on OpenTrials including the initial code used to scrape data from the EU Trials Register, Sarah McNeill for additional discussions on the structure of EUCTR data, Alex Walker for a code review on the Stata script, and Darren Smyth for guidance on European legislation and guidelines.

Web extra.

Extra material supplied by authors

Supplementary information: appendices 1 and 2

golb038037.ww1.pdf (661.4KB, pdf)

Contributors: BG conceived the project, designed the methods, conducted the analysis, wrote the paper, obtained funding, and supervised the project. FI built the website with input from SB, NJD, and BG. BG and NJD reviewed the legislation, with input from Darren Smyth. NJD extracted and processed the data in SQL, with input from BG, HC, SB, and JF. NJD normalised the sponsor names. BG, CH, NJD, and JF interpreted the findings. All authors contributed to and approved the final manuscript. BG is guarantor. This paper is compliant with the STROBE checklist. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: BG is funded by the Laura and John Arnold Foundation to conduct work on research integrity. No specific funding was sought for this project. The funder had no involvement in the study design or the decision to submit.

Competing interests: All authors have completed the ICMJE uniform competing interest form (available on request from the corresponding author) and declare: BG has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the NHS National Institute for Health Research (NIHR) School of Primary Care, the Health Foundation, NHS England, the NIHR Oxford Biomedical Research Centre, and the World Health Organization; he also receives personal income from speaking and writing for lay audiences on the misuse of science. CH has received grant funding from WHO, the NIHR, and the NIHR School of Primary Care. BG and CH are cofounders of the AllTrials campaign. NJD, SB, and FI are employed on BG’s grant from the Laura and John Arnold Foundation.

Ethical approval: Not required.

Data sharing: All data can be viewed at https://github.com/ebmdatalab/euctr-tracker-code. The analytic code and software are shared freely for re-use under an open licence and can be viewed at https://github.com/ebmdatalab/euctr-tracker-data.

Transparency: The lead author (BG) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

References

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary information: appendices 1 and 2

golb038037.ww1.pdf (661.4KB, pdf)

Articles from The BMJ are provided here courtesy of BMJ Publishing Group

RESOURCES