Moving Towards Transparency of Clinical Trials

An almost steady flow of articles have focused on the dangers or lack of efficacy of widely used drugs, along with allegations of hidden information, misinterpreted data, regulatory missteps, and corporate malfeasance. Many of these accounts involve analyses of research on human volunteers that had never been publicly disseminated,(1, 2) The uproar caused by an analysis of previously unpublished studies of the diabetes drug, Avandia, indicating that it may be harmful(3), is one recent example.(4–8) As a result, many question whether sufficient information about the safety and efficacy of medical interventions is available to the public(9), and whether society is meeting its ethical responsibilities to the human volunteers who put themselves at risk.(10, 11) Although advances in all areas of science depend upon free exchange of data, clinical trials warrant particular scrutiny due to their use of human volunteers and our dependence upon their results to inform medical decisions.

The persistent gap between the number of trials conducted and the number for which results are publicly available has been well-documented.(12, 13) Results may not be publicly disseminated for many reasons, ranging from lack of interest by authors or editors to publish results that seem uninteresting to outright attempts to hide “inconvenient” results.(14) A recent study suggests that over 30% of trials of 12 antidepressants submitted to the Food and Drug Administration (FDA) for review, primarily those with negative results, have not been published.(1) One effect of such “positive publication bias” is a boost in the apparent efficacy of these interventions. Trial registration policies, which mandate the public listing of basic trial information, and results database policies, which mandate submission and public posting of summary results within a certain timeframe(15), represent one type of response. However, improving transparency is only part of the solution to the broader set of concerns about medical interventions.

Promoting Transparency

Transparency exists along a continuum from documentation that a trial exists to full disclosure of the results data set at the end of the trial (Figure). “Trial registries” address one end of the spectrum by making public a summary of protocol details at trial initiation. “Results databases” provide public summaries of results for key trial endpoints, whether published or not. Some policies promote public access to data sets, such as the National Institutes of Health (NIH) Data Sharing Policy(16) and the Annals of Internal Medicine policy, which publishes author statements of willingness to share study protocols, statistical codes, and datasets.(17)

Figure.

While numerous clinical trial registries exist, the NIH has maintained ClinicalTrials.gov, the largest single registry of clinical trials, since 2000.(18) Although the law that led to the creation of ClinicalTrials.gov, the Food and Drug Administration Modernization Act of 1997, called for the registration of some trials of drug effectiveness for “serious or life-threatening diseases and conditions,”(19) other registration policies have encouraged broader voluntary registration of trial information. A policy by the International Committee of Medical Journal Editors (ICMJE) that requires prospective trial registration as a pre-condition for publication, effective September 2005(20), led to a 73% increase in trial registrations of all intervention types from around the world.(21) This increased rate of trial registration has stayed constant at about 250 new trials per week, resulting in nearly 44,800 trials from 150 countries as of September 2007.

Since 2005, the U.S. Congress has considered a number of bills calling for clinical trial registration and results reporting(22) and, on September 27, 2007, enacted the FDA Amendments Act (FDAAA)(23). Section 801 of this law (“FDAAA 801”) expands the scope of required registrations at ClinicalTrials.gov and provides for the first Federally-funded trial results database. It mandates registration of a set of controlled clinical investigations, other than phase I trials, of drugs, biologics, and devices subject to regulation by the FDA. The law applies to research for any condition regardless of sponsor type (e.g., industry, government, or academic). These new statutory requirements, though broader than previous law, remain narrower than the transnational policies of the ICMJE and the World Health Organization (WHO) which call for the registration of all interventional studies in human beings regardless of intervention type.(11, 24) FDAAA 801 also increases the number of mandatory data elements corresponding to the WHO and ICMJE international standard, and requires ClinicalTrials.gov to link the registry to specified, existing results information publicly available from the FDA Website, including summary safety and effectiveness data, public health advisories, and action packages for drug approval.

Importantly, FDAAA 801 also calls on the NIH to augment ClinicalTrials.gov to include a “basic results” database by September 2008. Data elements specified in the law include participant demographics and baseline characteristics; primary and secondary outcomes and statistical analyses; and disclosure of agreements between sponsors and non-employees restricting researchers from disseminating results at scientific forums. Generally, these data will be available to the public within 12 months of trial completion or within 30 days of FDA approval (or clearance) of a new drug, biologic, or device. The capacity to collect and display serious and frequent adverse event data observed during a trial is to be added to the system within two years.

Will FDAAA 801 Solve Recent Problems?

The Table illustrates a typology of public concerns about the system of evaluating drugs and devices. We have categorized selected recent controversial issues by alleged problem: Design, conduct, or analysis of the study; lack of public information about the study existence or results; and regulatory agency decision making.

Table.

Sample medical product safety concerns by category of perceived problems.

Category of Perceived Problem	Sample Issues	Recent Examples	Addressed by Section 801?
1. Design, Conduct, or Analysis, of clinical trial, including Ethical Issues
	Appropriateness of comparator Lack of data integrity/fraud Insufficient informed consent	Arcoxia(35), Ketek(34), Trovan(33)	No
2. Lack of Public Information about study:
a. Clinical Trial Existence/Results	Suppression of trial existence and results Attempted modification of pre-specified outcome measures	Paxil(40), Vioxx(30) Zetia(31	Yes
b. Observational Study Existence/Results	Study existence known only to company and results withheld	Baycol(39), Trasylol(36)	No
c. Post-market Adverse Event Reports Existence/Results	Failure to disseminate data	Ventak Prizm 2 DR Implantable Cardioverter- Defibrillator (ICD)(41)	Pending (e.g., postmarket surveillance is addressed in FDAAA, Section 905)
3. Regulatory Agency Decision-Making
	Delayed agency disclosure Delayed agency action	Ketek(34), Avandia(42)	No

FDAAA 801 directly addresses issues stemming from a lack of transparency in clinical trials, represented by the examples in the highlighted section of the Table. For instance, GlaxoSmithKline (GSK)-sponsored trial data for the heavily promoted antidepressant Paxil showing efficacy and safety concerns in children and adolescents were not available to the public.(9, 25, 26) The resulting 2004 legal settlement between GSK and the New York State Attorney General’s office required GSK to develop a publicly accessible online results database(27) for the timely, comprehensive, posting of results of company-marketed drugs to prevent similar incidents in the future.(28, 29) In the case of Vioxx, a COX-2 non-steroidal anti-inflammatory drug (NSAID) “several early, large clinical trials … were not published in the academic literature for years after Merck made them available to the FDA, preventing independent investigators from accurately determining its cardiovascular risk …(p.122)”(30) With evidence that Vioxx is associated with increased cardiovascular risk from subsequent clinical trials, the drug was voluntarily withdrawn from the market in September 2004.

More recently, questions have focused on the ENHANCE trial, an industry-sponsored study of Zetia, a marketed non-statin cholesterol-lowering drug that is also a component of the cholesterol-lowering drug Vytorin. Issues include delayed trial registration and results reporting, and attempting to modify pre-specified primary outcome measure.(31) Results, generally regarded as negative, were revealed in a company press release(32) following intense media attention and a Congressional investigation of these irregularities.(2)

Issues related to the design or conduct of clinical trials, including research ethics, are not covered by FDAAA 801. For example, allegations of human research protections violations in a 1996 Trovan (antibiotic) study on children in Zaire(33) and data integrity questions about Ketek (antibiotic) study 3014, in which FDA inspectors detected data fraud and other serious violations(34), would not be affected by FDAAA 801. Further, Merck studies of Arcoxia, a COX-2 NSAID, involving over 34,000 patients, were judged by the FDA to be of limited scientific interest because of the use of an “inappropriate” comparator with many known side effects of its own.(35) Nevertheless, it is possible that complete trial registration and results reporting might have helped institutional review boards (IRBs) assess the need for each additional Arcoxia trial.

While current policies have focused on interventional studies, observational studies play an increasingly critical role in biomedical research, especially in the assessment of safety after an intervention comes into widespread use. Postmarket observational studies provide data about rare, unanticipated adverse effects from the exposure of large numbers of heterogeneous individuals over periods of time longer than typically studied in controlled trials.(36) Despite methodological limitations such as susceptibility to confounding factors and other sources of bias which potentially lead to inconclusive or misleading results (e.g., data on risks of postmenopausal hormone replacement therapy from the Women’s Health Initiative(37)), observational research nevertheless can play a useful, complementary role to interventional studies.(38) Yet observational studies have received less attention in the quest for transparency.

In the case of a cholesterol-lowering drug Baycol, a company-conducted observational study showed a higher relative rate of muscle breakdown for Baycol compared with another marketed statin. This finding was never reported publicly, but became available as a result of litigation.(39) In another case, results of a Bayer-commissioned post-market observational study of 67,000 patients which raised concerns about associations between Trasylol, a clotting drug, and cardiovascular or renal risk, were not released by the company until after an FDA advisory committee meeting to evaluate the safety of Trasylol.(36, 40) In November 2007, Bayer announced the voluntary suspension of Trasylol sales worldwide pending further analysis of safety data. Some cases reflect a deficiency in monitoring serious adverse events once a drug or device is marketed. In other cases, such as Guidant’s Ventak Prizm 2 DR implantable cardioverter-defibrillator, information about a problematic adverse event profile (i.e., device malfunctions) was not publicly disseminated in a timely fashion(41).

Other concerns relate to thresholds for determining what and how much data are sufficient to prompt regulatory decisions regarding the availability or labeling of a medical product. For example, questions have been raised about the timeliness and adequacy of FDA’s response to data on Avandia(42), trials of Ketek(34), and on the use of some antidepressants in children(43, 44). Transparency policies alone do not address these issues, though they could help to empower members of the public who believe that regulatory action is warranted. For example, FDA mandated changes to the Avandia label in November 2007 to reflect the risk of myocardial infarction; this risk was first publicized in a meta-analysis published in June 2007(3) that used, in part, data from the GSK database mandated by the Paxil settlement.

Future Challenges

FDAAA 801 is intended to greatly expand the level of transparency for clinical trials, which could have a transformational effect on our system of evaluating drugs and devices. However, FDAAA 801 still leave areas of “opacity.” For example, although certain medical device trials must be registered at trial inception, this information is withheld from the public until device approval or clearance by FDA. The resulting “lockbox” prevents disclosure of trials of devices that are never approved or cleared (e.g., where safety concerns arose or sponsors abandoned further development). For instance, Boston Scientific stopped development of an experimental stent after clinical trials revealed frequent fractures in the device(45). In addition FDAAA 801 does not mandate public reporting for phase I drug trials and trials involving investigational interventions not regulated by the FDA, such as surgical procedures and behavioral therapies. Thus, lessons from phase I trials, such as the life-threatening adverse events in healthy volunteers caused by the super-monoclonal antibody, TGN1412, could go unreported to the public and, potentially, result in redundant studies by future unsuspecting researchers.(46) Further, results reporting is currently mandatory only for trials of FDA-approved medical products, allowing the results of unapproved products to remain hidden from public view.

Intellectual property-related issues

Current restrictions reflect a delicate balance between protection of commercial interests and promotion of public health.(15) Pharmaceutical, biotech, and medical device manufacturers are concerned that disclosures may undermine competitive advantage(47). However, there are important ethical and scientific reasons for broader disclosure: Trial participation by humans is predicated on the concept that the trial will add to “medical knowledge,” which requires dissemination of the results. In addition, it is not possible for a volunteer or an IRB to assess the risks and benefits of participation in a clinical trial if an unknown proportion of data on the proposed interventions is not publicly available.(48) FDAAA 801 calls on the Secretary of Health and Human Services to consider whether public health interests would support expanding the results requirements to include unapproved drugs, biologics, and devices, through a 3-year rulemaking process.

Validation of Information

Concerns have been raised about verifying the completeness or accuracy of sponsor- and researcher-submitted results data. The volume of completed trials (e.g., up to 200 new trials are expected per week based on extrapolation of ClinicalTrials.gov experience), the lack of access to protocols and data sets, and the subjective nature of some judgments are barriers to validation. The law reflects these concerns and mandates the reporting of objective data in tables. The NIH and FDA are directed by the law to conduct a “pilot quality control project” to inform the validation process. In addition, narrative summaries could be required in the future, but only if “such types of summaries can be included without being misleading or promotional.”(23) Additional research will be necessary to explore whether and how this might be accomplished.

Interpretation of Information

The results database will require an interface to assist users in finding study results. One concern is that members of the lay public and media may be ill-prepared to interpret summary results data.(49) Currently, there are no standards or guidelines for providing and explaining study results to trial participants, or to members of the public.(50) FDAAA 801 calls for the development of informational materials for consumers in consultation with health and risk communication experts. Furthermore, clinicians may be concerned that the existence of a results database will increase the patient expectations that clinicians will be knowledgeable about all results in the database, even those that were never published or disclosed in a journal. In addition, although the results database will facilitate the conduct of carefully conducted and comprehensive systematic reviews, some also worry that the public will have a difficult time assessing the quality of the multitude of analyses that may result.(51)

Conclusion

FDAAA 801 both expands the ClinicalTrials.gov registry and mandates a results database for trials of approved drugs and approved or cleared devices. Implementation of the law should transform the degree of public access to critical clinical trial information from publicly and privately funded clinical research. Drazen, Editor-in-Chief, New England Journal of Medicine, has noted that currently, some patients are “left on the cutting room floor to make a drug look better than it really is.”(52); FDAAA 801 should go a long way in ensuring that all patients and all data are publicly accounted for. Important policy insights and new practices for balancing transparency with other needs (e.g., intellectual property, validation, and interpretation) will be informed by the implementation and continued evolution of these policies. This goal will only be achieved through the participation and collaboration of all stakeholders. However, the trial registration and results reporting policies, while critical, cannot address all of the problems within the current system for ensuring safe and effective medical products.