Abstract
Background
Clinical trial registries are in widespread use to promote transparency around trials and their results.
Objective
To describe characteristics of drug trials listed in ClinicalTrials.gov and examine whether the funding source of these trials is associated with favorable published outcomes.
Design
An observational study of safety and efficacy trials for anticholesteremics, antidepressants, antipsychotics, proton-pump inhibitors, and vasodilators conducted between 2000 and 2006.
Setting
ClinicalTrials.gov, a Web-based registry of clinical trials launched in 1999.
Measurements
Publications resulting from the trials for the 5 drug categories of interest were identified, and data were abstracted on the trial record and publication, including timing of registration, elements of the study design, funding source, publication date, and study outcomes. Assessments were based on the primary funding categories of industry, government agencies, and nonprofit or nonfederal organizations.
Results
Among 546 drug trials, 346 (63%) were primarily funded by industry, 74 (14%) by government sources, and 126 (23%) by nonprofit or nonfederal organizations. Trials funded by industry were more likely to be phase 3 or 4 trials (88.7%; P < 0.001 across groups), to use an active comparator in controlled trials (36.8%; P = 0.010 across groups), to be multicenter (89.0%; P < 0.001 across groups), and to enroll more participants (median sample size, 306 participants; P < 0.001 across groups). Overall, 362 (66.3%) trials had published results. Industry-funded trials reported positive outcomes in 85.4% of publications, compared with 50.0% for government-funded trials and 71.9% for nonprofit or nonfederal organization–funded trials (P < 0.001). Trials funded by nonprofit or nonfederal sources with industry contributions were also more likely to report positive outcomes than those without industry funding (85.0% vs. 61.2%; P = 0.013). Rates of trial publication within 24 months of study completion ranged from 32.4% among industry-funded trials to 56.2% among nonprofit or nonfederal organization–funded trials without industry contributions (P = 0.005 across groups).
Limitations
The publication status of a trial could not always be confirmed, which could result in misclassification. Additional information on study protocols and comprehensive trial results were not available to further explore underlying factors for the association between funding source and outcome reporting.
Conclusion
In this sample of registered drug trials, those funded by industry were less likely to be published within 2 years of study completion and were more likely to report positive outcomes than were trials funded by other sources.
Primary Funding Source
National Library of Medicine and National Institute of Child Health and Human Development, National Institutes of Health.
ClinicalTrials.gov, a Web-based registry of clinical trials, was established in 1999 in response to increasing concerns about the lack of transparency in the conduct of clinical trials (1–4). The registry is a publicly available database of clinical trials and includes details on the types of interventions applied, primary outcomes measured, basic results of the trials, and organizations that funded the trials. Tracking the existence of clinical trials and a priori primary outcome measures provides the potential to monitor the selective publication of trials and trial results (5, 6).
Biased publication and reporting of results compromises the integrity of the medical literature by providing an incomplete account of the evidence (2, 7, 8). Several studies have highlighted widespread publication bias using documents submitted to drug licensing agencies, federal funding agencies, and scientific review boards (9 –13). As many as one half of trials supporting drugs that are approved by the U.S. Food and Drug Administration (FDA) remain unpublished 5 years after drug approval, and examination of unpublished results reveals striking discrepancies between existing trial results and results available in the published literature (10, 13).
The use of such registries as ClinicalTrials.gov has been embraced by the International Committee of Medical Journal Editors (ICMJE), and beginning in 2005, the ICMJE has required trial registration before participant enrollment as a prerequisite for publication in any of its member journals (14). In addition, the FDA Amendments Act of 2007 expanded the scope of ClinicalTrials.gov to augment comprehensive trial registration and ensure timely dissemination of trial results (15–17). However, little attention has been paid to another key element that may affect the presentation of clinical evidence and that is tracked in ClinicalTrials.gov: the funding source for clinical trials. We sought to describe characteristics of trials registered in ClinicalTrials.gov for selected groups of drugs and to examine whether the funding source of these trials is associated with published outcomes favorable to the drug studied.
Methods
Selection of Clinical Drug Trials
We identified all clinical trials registered in Clinical-Trials.gov as of 31 August 2009 that studied anticholesteremics, antidepressants, antipsychotics, proton-pump inhibitors, and vasodilators, including trials studying these drugs for FDA-approved and non–FDA-approved indications (Figure 1). We chose these drug classes because they represent well-defined groups of medications used to treat a broad spectrum of medical conditions. We identified the specific medications within these drug categories by using the classification in Lexi-Comp ONLINE (18), which classifies medications on the basis of their primary mechanism of action. We identified trials studying these drugs and selected those that started after 1 January 2000 and were completed by 31 December 2006. This approach allowed at least 3 years between trial completion and our literature search for publication of results. Trials that did not list a start date were excluded, and trials that did not list a completion date were included only if the trial status was updated to “complete” by the investigators before 31 December 2006. Finally, we included only safety or efficacy trials on the basis of the study description provided in the record. We defined efficacy trials as those studying the ability of a drug to produce a desired result and safety trials as those examining adverse effects associated with therapy under conditions of proposed use (19, 20).
Figure 1. Search and selection.
* 5 trials in the final trial group studied both an antidepressant and an antipsychotic.
Data From ClinicalTrials.gov
Data on characteristics of the trials and their registration were extracted from the ClinicalTrials.gov entry, including registration date, start date, completion date, condition treated, funding source, trial phase, primary outcome, anticipated enrollment number, age group of participants, and elements of the study design. Study completion date is defined by ClinicalTrials.gov as the date when participants are no longer being examined or treated (that is, the last patient’s final visit has occurred) (19). Each record was classified as including study results in the form of links to PubMed abstracts, links to unpublished result reports, or actual study results. All data were extracted independently by 2 authors, and disagreements were resolved by consensus.
The primary sponsor funding the study is classified in the registry as “NIH” (National Institutes of Health), “U.S. federal,” “other government,” “network,” “industry,” “individual,” and “other” (which includes universities, hospitals, foundations, and other nonprofit organizations). We categorized this variable into government (NIH, U.S. federal, and other government), industry, and nonprofit or nonfederal organizations (network, individual, and other). Secondary funding sources were categorized in the same manner. The registry categorizes age at which participants were eligible for enrollment as “child” (age ≤17 years), “adult” (age 18 to 65 years), “senior” (age ≥66 years), and combinations of these groups. We recoded the age group variable as children only, children and adults or seniors, and adults and/or seniors. Condition under study was classified as an FDA-approved or a non–FDA-approved indication on the basis of approval information provided on the drug label at the time of trial initiation (21).
Identification of Associated Trial Publications
We systematically searched 4 electronic databases for publications corresponding to the ClinicalTrials.gov trials. First, we searched PubMed by using names of principal investigators listed in the record and keywords for drug names, condition studied, design characteristics, study location, and outcome measures. We searched for publications in any language. If more than 1 publication was identified, we chose the publication that most closely fit the study description in the record. When a publication was not identified, we searched the Cochrane Library, EMBASE, and CINAHL databases by using a similar search strategy. We also searched an online result registry, ClinicalStudyResults.org, and result reports available through company Web pages for references to publications (22–24). For trials that remained unmatched, we attempted to contact the investigator or pharmaceutical company listed in the record with a maximum of 2 e-mails to inquire about publication status. All searches were updated and finalized as of 15 March 2010.
For each publication, we recorded the date of publication and the actual number of participants enrolled in the study. We examined the primary outcome and identified discrepancies with the primary outcome listed in the ClinicalTrials.gov record (5). We also examined results for the primary outcome and determined whether findings were favorable or not favorable to the test drug. Results were considered favorable if they were statistically significant (based on P values or CIs) and supported the efficacy or safety of the test drug or not favorable if they were not statistically significant for the efficacy or safety of the test drug (25). For noninferiority trials, if the test drug was equal to the comparison drug, the results were also classified as favorable. Publications that did not describe results pertaining to the safety or efficacy of the test drug (for example, early predictors of treatment response or effects of different dosing regimens) were classified as “neither.” Two authors independently did the literature search and data extraction, and disagreements were resolved by consensus. Intercoder agreement for assigning trial outcomes was excellent (κ = 0.92 [95% CI, 0.88 to 0.97]).
Statistical Analysis
We report the frequency of characteristics of the drug trials by primary funding source. We used chi-square tests to measure the association between funding source and trial characteristics and to compare characteristics before and after the ICMJE policy implementation. We used Cochran–Mantel–Haenszel tests to control for funding source when examining the effect of the ICMJE policy on trial registration and the effect of trial phase on reported outcomes. We examined changes in the rates of publication within 24 months of study completion by using a linear trend test for the 5 years between 2002 and 2006 (no trials were completed before 2002). We did a post hoc multivariate logistic regression analysis to assess the association between funding source and trial outcome, while controlling for other trial characteristics (drug class, approval status of indication, study phase, multicenter status, anticipated sample size, age of study population, comparator type, and length of study). We categorized funding source as industry, government, and nonprofit or nonfederal, except when we examined publication and results reporting characteristics, in which case trials with nonprofit or nonfederal funding were further categorized on the basis of the inclusion of industry funding. All data were analyzed with SAS software, version 9.1 (SAS Institute, Cary, North Carolina).
Role of the Funding Source
This research was funded by the National Library of Medicine and the National Institute of Child Health and Human Development, NIH. The funding sources had no role in the design of the study; collection, analysis, and interpretation of the data; preparation or review of the manuscript; or the decision to submit the manuscript for publication.
Results
We included 546 drug trials in the analysis, of which 346 (63%) were primarily funded by industry, 74 (14%) by government sources, and 126 (23%) by nonprofit or nonfederal organizations. Three (4%) trials funded by government sources and 61 (48%) trials funded by nonprofit or nonfederal sources received contributions from industry. For each of the drug classes, industry was the most common primary funding source, accounting for 39% to 89% of trials in each class (Figure 2). Of note, among the 346 industry-funded trials, 345 studied a drug developed or marketed by the company funding the trial.
Figure 2. Percentage of trials, by funding source within drug classes.
Industry was the primary funding source for all drug classes and funded more than two thirds of trials among 4 of the classes. Five trials studied both an antidepressant and an antipsychotic and are included in each class.
Study Characteristics
Study characteristics of drug trials varied on the basis of the primary funding source (Table 1). Trials funded by industry were more likely to be phase 3 or 4 trials (88.7%; P < 0.001 across groups), to use an active comparator in controlled trials (36.8%; P = 0.010 across groups), to be multicenter (89.0%; P < 0.001 across groups), and to enroll more participants (median sample size, 306 participants; P < 0.001 across groups). Government-funded trials were most likely to be placebo-controlled (56.2%), whereas trials funded by nonprofit or nonfederal sources were least likely to be multicenter (24.6%) and tended to have the smallest sample size (median, 50 participants). Industry-funded trials were also most successful at enrolling the anticipated number of participants, with 84.9% of trials enrolling at least 75% of the planned number of participants (P < 0.001 across groups). Only government-funded trials included a substantial number of children, with 37.8% of trials enrolling at least some pediatric patients (P < 0.001 across groups).
Table 1.
Characteristics of Drug Trials Registered in ClinicalTrials.gov
| Characteristic | Studies (n = 546) |
Primary Funding Source | ||
|---|---|---|---|---|
| Industry (n = 346) |
Government (n = 74) |
Nonprofit or Nonfederal (n = 126) |
||
| Indication FDA-approved, n (%)* | 295 (54.0) | 198 (57.2) | 32 (43.2) | 65 (51.6) |
|
Study phase, n (%)† Phase 1, 2, or 2/3 |
||||
| Phase 1, 2, or 2/3 | 88 (16.1) | 35 (10.1) | 26 (35.1) | 27 (21.4) |
| Phase 3 or 4 | 427 (78.2) | 307 (88.7) | 38 (51.4) | 82 (65.1) |
| Unknown Study design: interventional vs. observational, n (%) |
31 (5.7) | 4 (1.2) | 10 (13.5) | 17 (13.5) |
| Interventional | 540 (98.9) | 342 (98.8) | 73 (98.6) | 125 (99.2) |
| Comparator type in interventional trial (n = 540)‡ Active agent comparator |
180 (33.3) | 126 (36.8) | 17 (23.3) | 37 (29.6) |
| Placebo-controlled | 235 (43.5) | 132 (38.6) | 41 (56.2) | 62 (49.6) |
| No comparator Randomization in trials with a comparator (n = 415) |
125 (23.2) | 84 (24.6) | 15 (20.6) | 26 (20.8) |
| Yes | 408 (98.3) | 255 (98.8) | 58 (100) | 95 (96.0) |
| No | 6 (1.4) | 3 (1.2) | 0 | 3 (3.0) |
| Unknown | 1 (0.2) | 0 | 0 | 1 (1.0) |
| Blinding in trials with a comparator (n = 415)§ Double-blind |
337 (81.2) | 212 (82.2) | 50 (86.2) | 75 (75.8) |
| Single-blind | 8 (1.9) | 1 (0.4) | 2 (3.5) | 5 (5.1) |
| No blinding | 68 (16.4) | 45 (17.4) | 5 (8.6) | 18 (18.2) |
| Unknown | 2 (0.5) | 0 | 1 (1.7) | 1 (1.0) |
| Multicenter trial, n (%)† | 363 (66.5) | 308 (89.0) | 24 (32.4) | 31 (24.6) |
| Median anticipated sample size (IQR), n†‖ | 156 (60–432) | 306 (126–566) | 78 (40–150) | 50 (24–85) |
|
Final study sample ≥75% of anticipated, n (%)†¶ Age of study population, n (%)† |
278 (79.0) | 191 (84.9) | 23 (59.0) | 64 (72.7) |
| Children only | 30 (5.5) | 19 (5.5) | 1 (1.4) | 10 (7.9) |
| Children and adults | 47 (8.6) | 12 (3.5) | 27 (36.5) | 8 (6.4) |
| Adults only | 469 (85.9) | 315 (91.0) | 46 (62.2) | 108 (85.7) |
|
Length of study conduct, n (%)† <1 y |
120 (22.0) | 91 (26.3) | 8 (10.8) | 21 (16.7) |
| 1–2 y | 158 (28.9) | 113 (32.7) | 15 (20.3) | 30 (23.8) |
| >2 y | 209 (38.3) | 88 (25.4) | 50 (67.6) | 71 (56.4) |
| Unknown | 59 (10.8) | 54 (15.6) | 1 (1.4) | 4 (3.2) |
FDA = U.S. Food and Drug Administration; IQR = interquartile range.
Five drugs (representing 11 trials) were not FDA-approved. The indication for which the drug is currently under FDA review was used in these
P < 0.001 across variables for chi-square and Kruskal–Wallis tests for categorical and median values, respectively.
P = 0.010 for active comparator vs. placebo-controlled trials.
P = 0.02.
16 trials did not report sample size.
Final sample size was available only for the 362 trials with publications.
Trial Registration Practices
The funding source was also associated with characteristics of the ClinicalTrials.gov record (Table 2). Government-funded trials were more likely to be appropriately registered before the start date of the trial (25.7%), before study completion (75.3%), and before publication of the study (95.1%) (P < 0.001 across groups for all 3 variables). However, government-funded trials had the lowest proportion of reported primary outcomes (30.9%; P < 0.001 across groups). Provision of links to online abstracts of published results did not differ across funding sources, but industry-sponsored trials were most likely to provide links to unpublished results posted in study result registries or company Web sites. During the period studied, very few trials had begun including actual study results in the record.
Table 2.
Record Characteristics for Drug Trials Registered in ClinicalTrials.gov
| Characteristic and Category | Total, % (n/N) | Primary Funding Source, % (n/N) | Chi-Square P Value |
||
|---|---|---|---|---|---|
| Industry | Government | Nonprofit or Nonfederal |
|||
| Time of trial registration | |||||
| Before study start date | 9.5 (52/546) | 8.7 (30/346) | 25.7 (19/74) | 2.4 (3/126) | <0.001 |
| Before study completion date* | 48.7 (237/487) | 45.9 (134/292) | 75.3 (55/73) | 34.3 (48/122) | <0.001 |
| Before publication date | 79.6 (288/362) | 72.6 (167/230) | 95.1 (39/41) | 90.1 (82/91) | <0.001 |
|
Inclusion of primary outcome Primary outcome explicitly described in record† Provision of study results |
66.6 (158/237) |
72.4 (97/134) |
30.9 (17/55) |
91.7 (44/48) |
<0.001 |
| Link to published results available in record | 35.6 (129/362) | 33.9 (78/230) | 39.0 (16/41) | 38.5 (35/91) | 0.66 |
| Link to unpublished results available in record | 21.2 (39/184) | 33.6 (39/116) | 0 | 0 | NA |
| Trial results reported in record | 0.9 (5/546) | 1.4 (5/346) | 0 | 0 | NA |
NA = not applicable.
Completion date not reported for 59 trials.
Analysis performed for trials registered before study completion date.
Effect of ICMJE Policy on Registration Practices
In September 2004, the ICMJE announced a new policy requiring trial registration before patient enrollment for the trial to be eligible for publication in one of its member journals (14). Specifically, trials initiated before 1 July 2005 needed to be registered before 13 September 2005, and any trials initiated on or after 1 July 2005 required preregistration. Among ongoing trials as of 1 July 2005, 70.1% of industry-funded trials were compliant with the ICMJE policy, compared with 80.5% of government-sponsored trials and 60.0% of nonprofit or nonfederal organization–funded trials (P = 0.068 across groups). Of the 28 trials in our study that were initiated after the start date of the policy, 50% were appropriately preregistered, compared with 7.3% before this date (P < 0.001). Also, the number of trials registered before study completion increased, with an overall increase from 22.0% among trials completed before the policy began to 72.9% of trials completed after the policy took effect (P < 0.001, Cochran–Mantel–Haenszel test controlling for primary funding source). This improvement in timely registration occurred among trials in all funding categories, increasing from 18.9% to 73.6% among industry-funded trials, 68.8% to 80.5% among government-sponsored trials, and 1.9% to 67.1% among nonprofit or nonfederal source–funded trials.
Characteristics of Trial Publications
Overall, 66.3% of trials had published results (Table 3). Industry-funded trials resulted in the highest proportion of publications with favorable outcomes, with 85.4% of publications positive for the test drug compared with 50.0% of publications among government-funded trials and 71.9% of publications among nonprofit or nonfederal source–funded trials (P < 0.001). Because 48% of the trials primarily funded by nonprofit or nonfederal organizations also received some contributions from industry, this group was further categorized on the basis of industry support. This revealed a difference in the reported trial outcomes: 85.0% of trials funded by nonprofit or nonfederal sources with industry contributions reported favorable outcomes, compared with 61.2% of those without industry support (P = 0.013).
Table 3.
Publication and Results Reporting Characteristics of Drug Trials Registered in ClinicalTrials.gov
| Characteristic | Total, % (n/N) | Primary Funding Source, % (n/N) | Chi-Square P Value |
|||
|---|---|---|---|---|---|---|
| Industry | Government | Nonprofit or Nonfederal | ||||
| With Industry Contributions |
Without Industry Contributions |
|||||
| Published | 66.3 (362/546) | 66.5 (230/346) | 55.4 (41/74) | 67.2 (41/61) | 76.9 (50/65) | 0.065 |
| Primary outcome* | <0.001 | |||||
| Favorable | 78.3 (270/345) | 85.4 (188/220) | 50.0 (18/36) | 85.0 (34/40) | 61.2 (30/49) | |
| Not favorable | 21.7 (75/345) | 14.6 (32/220) | 50.0 (18/36) | 15.0 (6/40) | 38.8 (19/49) | |
| Primary outcome in publication inconsistent with registration† | 17.6 (15/85) | 8.7 (4/46) | 40.0 (4/10) | 17.6 (3/17) | 33.3 (4/12) | 0.046 |
| Published within 24 mo of study completion‡ | 39.7 (125/315) | 32.4 (60/185) | 53.7 (22/41) | 39.0 (16/41) | 56.2 (27/48) | 0.005 |
| Results from >1 trial pooled for publication | 8.6 (31/362) | 12.6 (29/230) | 0 | 0 | 4.0 (2/50) | 0.003 |
| Not published, but electronic results synopsis is available | 40.8 (75/184) | 64.7 (75/116) | 0 | 0 | 0 | NA |
NA = not applicable.
17 trials were classified as neither favorable nor unfavorable.
All trials included that were registered before completion date.
Completion date was not reported for 47 trials with publications.
Industry-funded trials were more likely to report positive findings across all trial phases, and phase was not a significant determinant of the reported outcome (P = 0.70, Cochran–Mantel–Haenszel test controlling for primary funding source) (Figure 3). In post hoc multivariate analyses, factors associated with the reporting of a favorable outcome in comparative trials included the primary funding source, whether the treatment indication was approved by the FDA, and whether the study was conducted at a single center.
Figure 3. Percentage of trials with reported favorable outcomes, by trial phase within funding source.
Trials funded by industry sources reported the highest percentage of favorable outcomes across all trial phases. Trial phase was not a significant determinant of outcome (P = 0.70, Cochran–Mantel–Haenszel test controlling for primary funding source). Phase 1 and 2 trials were combined because of the small number of phase 1 trials (n = 13).
Trials funded by nonprofit or nonfederal organizations that did not receive industry contributions were most likely (56.2%) and industry-funded trials least likely (32.4%) to be published within 24 months of study completion (P < 0.005 across groups). Trials funded by nonprofit or nonfederal organizations with industry contributions were also less likely than those without industry funding to be published within 24 months (39.0%). The proportion of trials published within 24 months of study completion increased between 2002 and 2006 after we controlled for primary funding source (increase from 9.5% to 54.4%; P < 0.001 for trend). This increase was among trials in each of the funding categories (from 7.7% to 44.7% among industry-funded trials, 25.0% to 68.8% among government-funded trials, and 0% to 60.0% among nonprofit or nonfederal organization–funded trials).
Discussion
Trial characteristics, registration practices, and publication patterns were related to funding source. Industry-funded trials were in more advanced phases of study and were most likely to be multicenter trials using an active comparator study design. Trials funded by government agencies had the highest rate of registration before the enrollment of participants but were less than half as likely to include an a priori primary outcome in their ClinicalTrials.gov study records. Industry-sponsored trials had the lowest rate of registration even before trial publication and the lowest rate of publication within 24 months of study completion.
Among the publications, those associated with trials primarily funded by industry were most likely to report a positive outcome. Trials primarily funded by nonprofit or nonfederal organizations that also received industry contributions were more likely than trials without such support to report findings favoring the study drug.
These results are consistent with previous studies demonstrating an association between industry funding and the reporting of results favoring experimental drugs marketed by the sponsoring company (25–33). Systematic reviews of the literature have shown that industry-funded trials are more than 4 times more likely to report findings supporting their product than are trials without financial interests (34). A study of head-to-head drug comparisons of statins manufactured by competing companies found that funding from the company marketing the statin under study was associated with a 16-fold increase in the likelihood of reporting results favoring the study drug (25). Other authors examining the source of this association have identified biases in trial design and data analyses, biased interpretation and reporting of results, and selective trial publication as possible causes (26, 32, 34–40).
We examined differences in trial characteristics based on funding source. Industry-funded trials, for example, tended to be in phase 3 or 4. Later-phase trials may be more likely to have positive outcomes because there is more certainty about the drug’s efficacy and safety at this advanced stage in the drug-development cycle. However, even after adjustment for trial phase, we found a relationship between funding source and positive outcome.
Other factors may contribute to this association. For instance, the choice of study question to be addressed in a trial is shaped by the intent of the investigation. Some trials are conducted to substantiate the efficacy and safety of a drug, whereas others are conducted to explore new indications. The study population selected may also affect the results, because it may more or less closely resemble the postmarketing target population. Furthermore, selection of the comparator treatment, drug doses, clinical end points, and duration of patient follow-up may all affect study conclusions.
Unknown or incomplete information about trial details makes factors that influence trial outcomes difficult to account for and measure (26, 32, 40). This limitation may be addressed in the future by proposed changes to ClinicalTrials.gov, such as incorporating study protocols and amendments into trial registrations and making FDA drug reviews public, including their statistical analyses and comprehensive summaries of trial results by independent experts (16, 33, 36, 38, 41, 42).
Although we could not assess all underlying factors, such as biases in trial design or data interpretation, we found that a substantial number of trials were not registered before the study start date or the publication of trial results. Without impartial, a priori registration of trials, planned data analyses and study outcomes may be altered or omitted in favor of the experimental treatment after trial data are available (33, 43, 44). The ICMJE’s requirement of trial preregistration as a condition for publication eligibility, effective in September 2005, has already resulted in a large increase in trial registration and will probably improve timely registration in the future (45).
This study demonstrates several strengths and weaknesses of using ClinicalTrials.gov to track factors leading to reporting of biased trial results. Trial start dates are easily assessed in the registry records and allow appraisal of preregistration of trials, which is essential if the registry is to be used to screen for outcome reporting bias (5, 38). Any changes made to the record after initial submission are tracked, and earlier versions are archived and accessible in the record (46). In our study sample, however, fewer than 10% of trials were registered before the study start date. Concern remains that certain data, including descriptions of primary outcomes, are insufficiently detailed, but the use of these trial records already allows identification of inconsistencies between data elements reported in the registry and those reported in the final article (5, 6).
Another benefit of the registry is that records indicate whether a trial is completed and provide investigators with data fields to include references to resulting publications. Use of unique clinical trial registry numbers in publications allows automated identification of trial publications in MEDLINE and addition of references to the ClinicalTrials.gov record (46). However, only about one third of published trials contained a link to a publication in the record, and manual searches were required to determine whether the remaining trials had been published. This finding is not surprising, considering that a recent study found that 37% of journals did not provide guidelines on registration requirements in instructions to authors and 27% of published trials were not registered (5). To maximize the value of clinical trial registries for monitoring and reducing publication bias, publication status must be more consistently tracked in registration records and publications must contain registration information.
A limitation of our study is that we could not confirm publication status for all trials. Some trials may be incorrectly classified as unpublished, thus overestimating the degree of publication bias. However, the search methods were rigorous, and publication status is unlikely to be misclassified in a systematic manner based on funding source. Another limitation is that we allowed a minimum of 3 years between study completion and publication, which may not have always been sufficient time for publication. Again, this is unlikely to have affected our findings of the effect of funding sources on registration practices or results reporting. Our approach to assessing whether trial findings were favorable was systematic and consistent. However, publications were classified as reporting favorable outcomes based on the statistical significance of results reported in the publication; without further information on the quality of the trial design and conduct and the analytic methods used, we cannot verify definitively whether the drug was truly effective or safe. Also, many of the data fields in ClinicalTrials.gov have missing values.
We show a positive association between industry funding and reporting of favorable outcomes among drug trials registered in ClinicalTrials.gov. Industry-sponsored trials were also less likely to be published within 24 months of study completion. Investigators of trials funded by either for-profit or nonprofit sources often did not comply with best practices around trial registration and results publication, which may lead to a biased knowledge base. For trial registries to maximize transparency around clinical trials and ensure the validity and quality of the resulting scientific evidence, timely, accurate, and expanded information entry and results disclosure are required.
Context
ClinicalTrials.gov is a publicly available, Web-based registry that helps identify and monitor clinical trials.
Contribution
This observational study of a sample of trials registered in ClinicalTrials.gov reports the following funding sources for 546 selected drug trials: industry (63%), government (14%), and nonprofit or nonfederal organizations (23%). Trials funded by industry more often were phase 3 or 4 trials, used an active comparator, were multicenter, enrolled more participants, were not published within 2 years of completion, and reported favorable outcomes when published than did trials funded by other sources.
Caution
Detailed information on study conduct and quality were not assessed.
—The Editors
Acknowledgments
Grant Support: By training grant 5T32HD040128 (Dr. Bourgeois) from the National Institute of Child Health and Human Development and grant 5G08LM009778 (Dr. Mandl) from the National Library of Medicine.
Footnotes
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-0087.
Reproducible Research Statement: Study protocol: Not available. Statistical code: Available from Dr. Bourgeois (e-mail, florence.bourgeoischildrens.harvard.edu). Data set: Primary data are available at ClinicalTrials.gov and PubMed.
Current author addresses and author contributions are available at www.annals.org.
Author Contributions: Conception and design: F.T. Bourgeois, S. Murthy, K.D. Mandl.
Analysis and interpretation of the data: F.T. Bourgeois, S. Murthy, K.D. Mandl.
Drafting of the article: F.T. Bourgeois, S. Murthy, K.D. Mandl.
Critical revision of the article for important intellectual content: F.T. Bourgeois, S. Murthy, K.D. Mandl.
Final approval of the article: F.T. Bourgeois, S. Murthy, K.D. Mandl. Statistical expertise: F.T. Bourgeois.
Obtaining of funding: F.T. Bourgeois, K.D. Mandl.
Administrative, technical, or logistic support: S. Murthy.
Collection and assembly of data: F.T. Bourgeois, S. Murthy.
References
- 1.Horton R, Smith R. Time to register randomised trials. The case is now unanswerable [Editorial] BMJ. 1999;319:865–866. doi: 10.1136/bmj.319.7214.865. [PMID:10506022] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4:1529–1541. doi: 10.1200/JCO.1986.4.10.1529. [PMID:3760920] [DOI] [PubMed] [Google Scholar]
- 3.Tonks A. Registering clinical trials. BMJ. 1999;319:1565–1568. doi: 10.1136/bmj.319.7224.1565. [PMID:10591727] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Yamey G. Scientists who do not publish trial results are “unethical”. BMJ. 1999;319:939A. doi: 10.1136/bmj.319.7215.939a. [PMID:10514148] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302:977–984. doi: 10.1001/jama.2009.1242. [PMID:19724045] [DOI] [PubMed] [Google Scholar]
- 6.Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.gov: a cross-sectional analysis. PLoS Med. 2009;6 doi: 10.1371/journal.pmed.1000144. e1000144. [PMID:19901971] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990;263:1405–1408. [PMID:2304220] [PubMed] [Google Scholar]
- 8.Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124. doi: 10.1371/journal.pmed.0020124. [PMID:16060722] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330:753. doi: 10.1136/bmj.38356.424606.8F. [PMID:15681569] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med. 2008;5:e191. doi: 10.1371/journal.pmed.0050191. [PMID:18816163] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008;5:e217. doi: 10.1371/journal.pmed.0050217. [PMID:19067477] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chan AW, Krleza-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171:735–740. doi: 10.1503/cmaj.1041086. [PMID:15451835] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–260. doi: 10.1056/NEJMsa065779. [PMID:18199864] [DOI] [PubMed] [Google Scholar]
- 14.DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors [Editorial] JAMA. 2004;292:1363–1364. doi: 10.1001/jama.292.11.1363. [PMID:15355936] [DOI] [PubMed] [Google Scholar]
- 15.U.S. Food and Drug Administration Amendments Act of 2007. 2007:105–185. Pub. L. No. [Google Scholar]
- 16.Wood AJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med. 2009;360:824–830. doi: 10.1056/NEJMsr0806582. [PMID:19228628] [DOI] [PubMed] [Google Scholar]
- 17.Zarin DA, Tse T. Medicine. Moving toward transparency of clinical trials. Science. 2008;319:1340–1342. doi: 10.1126/science.1153632. [PMID:18323436] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lexi-Comp ONLINE. [Accessed on 20 May 2010]; Available at http://online.lexi.com.
- 19. ClinicalTrials.gov Glossary of Clinical Trials Terms. [Last updated 18 March 2008];2010 May 20; Accessed at http://clinicaltrials.gov/ct2/info/glossary on.
- 20.O’Connor AB. Building comparative efficacy and tolerability into the FDA approval process. JAMA. 2010;303:979–980. doi: 10.1001/jama.2010.257. [PMID:20215612] [DOI] [PubMed] [Google Scholar]
- 21.U.S. Food and Drug Administration. [Accessed on 20 May 2010];Drugs@FDA. Available at www.accessdata.fda.gov/scripts/cder/drugsatfda/
- 22.AstraZeneca Clinical Trials. [Accessed on 20 May 2010]; Available at www.astrazenecaclinicaltrials.com.
- 23.Bristol-Myers Squibb Clinical Trial Registry. [Accessed on 20 May 2010]; Available at www.bms.com/clinical_trials.
- 24.GlaxoSmithKline Clinical Study Register. [Accessed on 20 May 2010]; Available at www.gsk-clinicalstudyregister.com.
- 25.Bero L, Oostvogel F, Bacchetti P, Lee K. Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others. PLoS Med. 2007;4:e184. doi: 10.1371/journal.pmed.0040184. [PMID:17550302] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290:921–928. doi: 10.1001/jama.290.7.921. [PMID:12928469] [DOI] [PubMed] [Google Scholar]
- 27.Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289:454–465. doi: 10.1001/jama.289.4.454. [PMID:12533125] [DOI] [PubMed] [Google Scholar]
- 28.Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004;170:477–480. [PMID:14970094] [PMC free article] [PubMed] [Google Scholar]
- 29.Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, et al. The uncertainty principle and industry-sponsored research. Lancet. 2000;356:635–638. doi: 10.1016/S0140-6736(00)02605-2. [PMID:10968436] [DOI] [PubMed] [Google Scholar]
- 30.Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325:249. doi: 10.1136/bmj.325.7358.249. [PMID:12153921] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Nkansah N, Nguyen T, Iraninezhad H, Bero L. Randomized trials assessing calcium supplementation in healthy children: relationship between industry sponsorship and study outcomes. Public Health Nutr. 2009;12:1931–1937. doi: 10.1017/S136898000900487X. [PMID:19254426] [DOI] [PubMed] [Google Scholar]
- 32.Ridker PM, Torres J. Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000–2005. JAMA. 2006;295:2270–2274. doi: 10.1001/jama.295.19.2270. [PMID:16705108] [DOI] [PubMed] [Google Scholar]
- 33.Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–1971. doi: 10.1056/NEJMsa0906126. [PMID:19907043] [DOI] [PubMed] [Google Scholar]
- 34.Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326:1167–1170. doi: 10.1136/bmj.326.7400.1167. [PMID:12775614] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Avorn J. Keeping science on top in drug evaluation. N Engl J Med. 2007;357:633–635. doi: 10.1056/NEJMp078134. [PMID:17699813] [DOI] [PubMed] [Google Scholar]
- 36.Chan AW. Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med. 2008;5:e230. doi: 10.1371/journal.pmed.0050230. [PMID:19067481] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Johansen HK, Gotzsche PC. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. JAMA. 1999;282:1752–1759. doi: 10.1001/jama.282.18.1752. [PMID:10568648] [DOI] [PubMed] [Google Scholar]
- 38.O’Connor AB. The need for improved access to FDA reviews. JAMA. 2009;302:191–193. doi: 10.1001/jama.2009.973. [PMID:19584349] [DOI] [PubMed] [Google Scholar]
- 39.Safer DJ. Design and reporting modifications in industry-sponsored comparative psychopharmacology trials. J Nerv Ment Dis. 2002;190:583–592. doi: 10.1097/00005053-200209000-00002. [PMID:12357091] [DOI] [PubMed] [Google Scholar]
- 40.Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med. 2005;2:e138. doi: 10.1371/journal.pmed.0020138. [PMID:15916457] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Lassere M, Johnson K. The power of the protocol. Lancet. 2002;360:1620–1622. doi: 10.1016/S0140-6736(02)11652-7. [PMID:12457782] [DOI] [PubMed] [Google Scholar]
- 42.Miller JD. Registering clinical trial results: the next step. JAMA. 2010;303:773–774. doi: 10.1001/jama.2010.207. [PMID:20179288] [DOI] [PubMed] [Google Scholar]
- 43.Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–2465. doi: 10.1001/jama.291.20.2457. [PMID:15161896] [DOI] [PubMed] [Google Scholar]
- 44.Chan AW, Hróbjartsson A, Jørgensen KJ, Gøtzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008;337:a2299. doi: 10.1136/bmj.a2299. [PMID:19056791] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Zarin DA, Tse T, Ide NC. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med. 2005;353:2779–2787. doi: 10.1056/NEJMsa053234. [PMID:16382064] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Zarin DA, Ide NC, Tse T, Harlan WR, West JC, Lindberg DA. Issues in the registration of clinical trials. JAMA. 2007;297:2112–2120. doi: 10.1001/jama.297.19.2112. [PMID:17507347] [DOI] [PubMed] [Google Scholar]