Should preclinical studies be registered?

To the Editor

In recent years, numerous journals, regulators and ethics guidelines have adopted policies for prospective registration of clinical trials and full reporting of results. Whereas at one time, investigators published controlled clinical trials at their discretion, policies now compel prospective registration and deposition of trial results—including those that are negative and inconclusive—in publicly accessible databases¹. In May, an editorial in Nature Biotechnology also supported the full disclosure of trial results in response to a call for more openness from European regulators². In the following correspondence, we examine whether arguments that justify trial registration also hold for preclinical studies aimed at supporting trials. We argue that the high rate of failure in translating basic research into medical therapies raises concerns about human protections and inefficiencies in the research enterprise; registration and reporting of preclinical studies would partly address these concerns. We close by suggesting that funding agencies and others establish working groups to explore mechanisms for reconciling preclinical registration with the strategic imperatives in drug development.

When registration and public deposition of results (‘good disclosure practice’) were canonized as ethical principles, two main arguments were offered. First, good disclosure practice respects the altruism of human subjects by helping ensure that studies see the light of day. Publication is the first step in a process through which findings in isolated settings are synthesized into collective knowledge. If this process is interrupted by nonpublication, burdens endured by human volunteers lose their moral justification. The second argument rested on protecting downstream users—patients and institutions whose interests are bound up with an unimpeded flow of findings. Absent good disclosure practice, researchers coming forward with unfavorable findings are at a reputational and funding disadvantage relative to those withholding them. This incentivizes biased reporting. Biased reporting potentially harms patients by distorting treatment recommendations. It also thwarts the ability of healthcare systems to allocate resources in accordance with the best evidence.

Various commentators have made a case that a version of the first argument extends to preclinical studies³ (by which we mean in vivo toxicology and hypothesis-testing experiments expressly aimed at demonstrating biological activity against disease and designed to support clinical translation). Most commentators on ethics recognize that animals have moral status. The burdens imposed on animals in medical experiments are arguably unredeemed if the knowledge is never shared with the broader research community.

As attractive as this justification may be, an argument more aligned with contemporary policy priorities can be made with the user argument: preclinical disclosure helps secure the welfare of individuals and institutions that rely on access to evidence. This position gains force when set against the high rates of attrition during human testing. One study found that only 11% of new products that enter phase 1 trials are licensed; for cancer and neurological disorders, the figures are closer to 5% and 8%, respectively⁴. Various longitudinal studies show that most highly promising preclinical findings resist translation. These failures and heavy expenditures should prompt a search for strategies that help ensure volunteers are not needlessly enrolled in trials, and that scarce research resources are not squandered.

The most proximate consumers of preclinical evidence are investigators, regulators and review bodies charged with ensuring that early-phase trial risks are favorably balanced with benefits. Good disclosure helps these parties protect volunteers³. Many early-phase trials involve substantial burdens and risks. Phase 1 trials involving healthy subjects, by definition, set out to cause toxicities, and those involving cancer or surgical delivery are considerably more burdensome. Initiation of trials should therefore be guided by a careful assessment of preclinical evidence. Translational trials are only warranted insofar as there is justified belief in the agent’s clinical promise. Selective publication frustrates appraisal of promise by limiting the ability of decision makers to assess the totality of preclinical evidence. It also can lead to overestimating treatment effects, as indicated by recent studies showing inflation of effect sizes by 30% due to publication bias³.

The consequences of selective preclinical publication propagate to later phases of testing. Phase 1 trials are generally not designed to test efficacy. Therefore, phase 2 and 3 trials, which enroll hundreds of patients, are heavily informed by the biological activity observed in preclinical studies. As many as 45% of drugs fail in late stages of development, with insufficient activity being the leading cause of attrition³. Good disclosure helps secure the justification for exposing larger numbers of volunteers to unproven agents.

A second community of downstream users of preclinical evidence is the institution of biomedical research as a whole. Research communities benefit from the free flow of scientific information. Lack of timely access to preclinical observations hampers researchers’ ability to wrest valuable insights from regular failures in clinical development.

Good disclosure addresses this in two ways. First, preclinical results provide an important resource for brokering competing explanations when agents fail translation. Interpretation of experimental findings depends on observations external to a given experiment⁵. Trials that do not meet their endpoints might be interpreted as disconfirming clinical promise of an intervention, or they might indicate flawed trial design. If a review finds preclinical studies were rigorously designed, failure to confirm activity in human beings might plausibly be attributed to flaws in trial conduct rather than spurious signal in animals. For instance, retrospective analysis of AstraZeneca’s unsuccessful stroke drug, NXY059, indicated the free-radical trapping drug did not show activity in hypertensive rats⁶; this finding may help explain trial results, given that most stroke patients are hypertensive. Second, disclosure of animal findings enables secondary analyses that advance knowledge about translation itself. Retrospective analysis of pooled preclinical studies have enabled researchers to address questions like, Are anti-sepsis candidates more promising in certain types of sepsis⁷? Should toxicology studies of chronic-use agents run longer than six months⁸? Which preclinical techniques best predict human pharmacokinetics for small-molecule drugs⁹? Does transgene copy number in mouse models confound effect sizes?

We anticipate three objections to good preclinical disclosure practice. First, it jeopardizes the strategic advantage of drug developers by enabling free-riding on investments in clinical development. A similar argument was initially made for excluding phase 1 trials from trial registries. We agree that obligations should cohere with the current system of private drug development. We suggest that private interests might be reconciled with good disclosure practices by attending to policy objectives in light of five key questions in registry or data repository architecture. (i) Which mechanisms should be used to incentivize deposition? (ii) Which actors should have access to registries or repositories? (iii) At what stage should actors gain access? (iv) How much detail about experimental design should be included? (v) What outcomes should be deposited? For instance, if policymakers prioritize the protection of human subjects, then repository access might be restricted to regulatory authorities and ethics review committees. Or, public access to preclinical results might be postponed until completion of phase 2 trials.

A second objection is the problem of regress. If the logic supporting good disclosure practice extends to preclinical studies, does it extend to basic science as well? If so, does this create an unacceptable burden for researchers? We suggest delimiting our proposal to controlled invivo animal studies directed at testing disease response and toxicity, or toxicology studies in live animals. These studies bear more immediately on the welfare of subjects and the interpretability of trials. Moreover, these studies have a structure similar to controlled clinical trials and are unlike basic science (which has a more exploratory structure).

Third is the question of cost. Registries entail administrative costs (the 2007 budget for clinicaltrials.gov (http://clinicaltrials.gov/) was $3.0 million) and compliance expense for investigators. Well-documented flaws in reporting and compliance with trial registries would likely be recapitulated in preclinical registries¹⁰. Would benefits outweigh costs? Without a proper cost-benefit analysis or a specified registry architecture, we cannot provide an unequivocal answer. We do not underestimate the technical, economic, political and logistical challenges of incentivizing good preclinical disclosure. Nevertheless, models less costly than clinicaltrials.gov might be explored, such as those used to promote deposition of genomic and microarray data. High-impact biomedical journals, for example, might encourage good disclosure practices by requiring authors of trials or preclinical experiments to include a sentence stating that complete summary preclinical evidence has been deposited in a public database. Funding agencies might consider mechanisms, like those used for prospective protocol review in gene transfer trials, to encourage researchers at publicly funded institutions to deposit supporting preclinical evidence.

Trial registries began with a series of modest steps that afforded opportunities to test and refine models. Initially, registries incorporated only publicly sponsored studies. Research agencies and drug regulators then convened working groups to establish data elements for inclusion. Only later did journals and legislation establish rules that significantly incentivized good disclosure practice¹¹. Implementation of preclinical registries should follow a similar course of iterative refinement.

In recent years, various funding bodies have signaled their commitment to clinical translation by creating new support and mechanisms for research. To better meet the potential of these programs while enhancing protections for volunteers, we urge funding agencies, journals, foundations and academic institutions to devise policies that promote registration and reporting of preclinical results.