RESEARCH LETTER
Pivotal efficacy trials, trials that form the basis of the Food and Drug Administration’s (FDA) decision to approve a novel therapeutic agent,1 have great relevance to clinical practice because when these therapies are first approved for use, few clinical trials have been conducted. However, studies focused on pharmacotherapies approved prior to 2005 found that more than one-quarter of these trials were not published,2,3 although efforts to promote clinical trial transparency and dissemination have since intensified.4 We determined publication status and timeliness among pivotal trials supporting the approval of novel therapeutics by FDA between 2005 and 2011.
METHODS
As described previously,1 we assembled a cohort of pivotal trials for all novel pharmacologics and biologics approved by FDA between 2005 and 2011. Trials were categorized by therapeutic and trial characteristics, including randomization, allocation strategy, duration, and number of patients (Table).
TABLE.
Publication of pivotal trials supporting the approval of novel therapeutics by the Food and Drug Administration between 2005 and 2011, stratified by novel therapeutic and trial characteristics.
| Novel Therapeutic and Trial Characteristics | Published, No. (%) | P Value* |
|---|---|---|
| Overall (n=380) | 326 (86) | |
| Approval Year | 0.48 | |
| 2005–2008 (n=193) | 168 (87) | |
| 2009–2011 (n=187) | 158 (85) | |
| Novel Therapeutic Type | 0.88 | |
| Pharmacologic (n=321) | 275 (86) | |
| Biologic (n=59) | 51 (86) | |
| Therapeutic Area | 0.003 | |
| Cardiovascular Disease, Diabetes Mellitus and Hyperlipidemia (n=72) | 55 (76) | |
| Infectious Disease (n=54) | 52 (96) | |
| Psychiatry (n=43) | 37 (86) | |
| Oncology (n=42) | 39 (93) | |
| Neurology (n=34) | 32 (94) | |
| Autoimmune & Musculoskeletal (n=29) | 29 (100) | |
| Dermatology (n=25) | 17 (68) | |
| Other (n=81) | 65 (80) | |
| Trial Randomized | 0.23 | |
| Yes (n=341) | 295 (87) | |
| No (n=39) | 31 (79) | |
| Trial Allocation Strategy | 0.28 | |
| No blinding (n=40) | 32 (80) | |
| Single blind (n=9) | 9 (100) | |
| Double blind (n=302) | 262 (87) | |
| N/A (single arm) (n=29) | 23 (79) | |
| Trial Duration | 0.29 | |
| < 12 weeks (n=165) | 137 (83) | |
| 12 – 24 weeks (n=124) | 111 (90) | |
| > 24 weeks (n=91) | 78 (86) | |
| Total Number of Trial Patients | 0.02 | |
| < 200 patients (n=96) | 77 (80) | |
| 200 – 400 patients (n=77) | 62 (81) | |
| >400 patients (n=207) | 187 (90) |
P value for Pearson’s chi-square testing of differences in publication status by therapeutic and trial characteristics.
To determine trial publication status, two reviewers (JWS, NSD) independently searched the biomedical literature between April and October 2013 using a systematic strategy. Publications were identified in one of two ways: we searched the Scopus database (Elsevier, Inc.; Philadelphia, PA) using the terms “[generic drug name]” AND “clinical trial”. Alternatively, manufacturer-designated trial identification numbers of six or more characters were entered into the Advanced Search feature of ClinicalTrials.gov.5 We used four criteria to identify matching publications: study design, indication, intervention, and intention-to-treat enrollment; publication enrollment was allowed to exceed that reported by FDA as enrollment often extends beyond regulatory submission. Publications that pooled results from multiple trials were credited to all relevant trials if trial design was discussed in detail. For all unpublished trials, a third reviewer (JSR) independently confirmed publication status.
To determine trial publication timeliness, we calculated time from approval to publication in months for all trials which were published. We examined associations between publication and both therapeutic and trial characteristics using Pearson’s chi-square analysis, performed with JMP 10.0.0 software (SAS Institute, Inc.; Cary, NC).
RESULTS
There were 380 pivotal efficacy trials associated with 149 novel therapeutics approved for 164 indications by FDA; median number of trials per indication was 2. Overall, 86% of pivotal trials (326 of 380) were published in peer-reviewed biomedical journals. Among 164 indications, at least one pivotal trial was published for 96% (157 of 164), whereas all pivotal trials were published for 82% (134 of 164). Among all published trials, 48% (157 of 326) were published before or during the month of FDA approval; among trials published after approval, median time from approval to publication was 7.0 months (range: 1–73). Finally, there were significant differences in publication rates by therapeutic area (p=0.004) and by total number of patients (p=0.02), but not by other characteristics (Table).
DISCUSSION
Among the pivotal efficacy trials for all novel therapeutics approved by FDA between 2005 and 2011, 86% were published in the peer-reviewed biomedical literature, although timeliness varied, slightly more than the 76% published for drugs approved between 1998 and 2000 and the 69% for anti-depressants approved between 1987 and 2004, although the former used a slightly different search strategy and the latter included both pivotal and supporting trials.2,3 Nevertheless, our findings suggest there is opportunity for improvement, particularly because these trials are critical to FDA approval decisions and directly relevant to clinical practice as there is otherwise limited clinical research available to inform physician and patient decision-making when therapies are first approved for use.
Our study was limited to pivotal trials; we did not examine whether the many supporting trials submitted as part of novel therapeutic applications were published, although we expect their publication rates to be lower. In addition, we did not contact manufacturers to confirm that trials were unpublished, nor did we determine whether unpublished trials reported results on ClinicalTrials.gov.
Acknowledgments
Data access and responsibility: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding/support and role of the sponsor: This project was not supported by any external grants or funds. Dr. Ross receives support from Medtronic, Inc. and Johnson and Johnson, Inc. to develop methods of clinical trial data sharing, from the Centers for Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting, and from the Food and Drug Administration (FDA) to develop methods for post-market surveillance of medical devices. Dr. Ross is supported by the National Institute on Aging (K08 AG032886) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program.
Footnotes
Conflicts of interest: Mr. Smithy reports that he owns a small number of shares of the following pharmaceutical companies: small number of shares of Bristol Myers Squibb, Gilead, Johnson and Johnson, and Roche. Dr. Ross reports that he is a member of a scientific advisory board for FAIR Health, Inc.
Author contributions: Mr. Smithy drafted the manuscript and conducted the statistical analysis. Dr. Ross was responsible for the conception and design of this work, obtained funding, and provided supervision. All authors were responsible for acquisition of data, participated in the analysis and interpretation of the data, and critically revised the manuscript for important intellectual content.
References
- 1.Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005–2012. JAMA. 2014;311(4):368–377. doi: 10.1001/jama.2013.282034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lee K, Bacchetti P, Sim I. Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis. PLoS Med. 2008;5(9):e191. doi: 10.1371/journal.pmed.0050191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–260. doi: 10.1056/NEJMsa065779. [DOI] [PubMed] [Google Scholar]
- 4.Ross JS, Gross CP, Krumholz HM. Promoting Transparency in Pharmaceutical Industry-Sponsored Research. Am J Public Health. 2012;102(1):72–80. doi: 10.2105/AJPH.2011.300187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.U.S. National Library of Medicine, U.S. National Institutes of Health, U.S. Department of Health and Human Services. ClinicalTrials.gov: Advanced Search. Available at: http://clinicaltrials.gov/ct2/search/advanced. Accessed October 30, 2013.