Registered animal studies and null data

Abstract

EMBO Press encourages to document or pre‐register animal experiments on the new Animal Study Registry to increase reproducibility and efficacy of animal research.

Only a fraction of what is produced in biomedical research is archived, let alone shared. Science can be a finicky business, and there are limited reasons for chronicling and sharing the results of experiments gone wrong. Yet, that is a long way from publishing only highly selected—if not cherry‐picked—results in narrative‐driven research papers. Many—maybe most—experimental data in the biosciences end up not supporting the original hypothesis. That is rather reassuring, as in good Popperian tradition scientists should do all they can to challenge their hypotheses before publishing them as a new finding (immortalized by Thomas Henry Huxley's 1870 axiom “The great tragedy of science—the slaying of a beautiful hypothesis by an ugly fact”). However, not sharing otherwise solid and controlled “negative” data—better referred to as “null data” to avoid any misunderstanding that “negative” means antithetical or “bad”—does the community a disservice and wastes funds by leaving it open for others to repeat such futile experiments. Equally, it is often hard to find a journal to accept data that contradict—or appears to contradict—previous results, conclusions or even “dogma”, potentially side‐stepping the crucial tempering of conclusions that remain unchallenged. Results based on flawed experimental design can mislead and detract (even if it can certainly be useful to learn why a certain experimental setup turned out to be flawed), but inconclusive data are not a priori worthless. Scientists should strive to archive and share all well‐executed results in a findable, accessible, interoperable and reusable (FAIR) manner.

Animals in the lab

Animal experimentation is a crucial component of biomedical research. The power of animal models has been greatly increased by stunning technological advances around imaging and surgically accurate gene editing. From high‐resolution imaging of Drosophila development to intravital imaging of tumour microenvironment or metastization in mouse, it is clear that understanding the true complexity of nature requires extending in vitro and ex vivo observation to the organismal context. Nonetheless, it is also necessary that research on animals is regulated to minimize their suffering. While regulation for research on vertebrates can be perceived as a burden in many countries, it is essential that appropriate checks and balances are applied, independently monitored and reported—EMBO Press journals encourage (https://www.embopress.org/page/journal/14693178/authorguide#animalwelfare) such reporting following the ARRIVE guidelines (https://www.nc3rs.org.uk/arrive-guidelines). We receive occasionally research papers that appear to overstep international norms of animal research ethics, and our journals have appointed expert advisory board members to help adjudicate on the often finicky decisions of what are acceptable norms in animal and human researches.

Sharing is caring

The 3R concept (reduce, refine, replace) provides a well‐established framework for planning animal experiments. However, reducing and refining animal experimentation can only be achieved if the research community shares all meaningful experiments. This has to include descriptive data, null data and data contradicting previous claims. Scientists increasingly use preprints that can readily capture such data, as do platforms such as BioStudies, FigShare and Dryad as well as journals that select for quality alone, such as PloS ONE and Scientific Reports. However, what has been missing are platforms specifically designed to capture details on animal experiments in a structured manner, ideally before they are done. This has now been resolved: in this issue, Bert and colleagues report on the first open repository for animal experiments (http://www.animalstudyregistry.org) 1. The Animal Study Registry (http://www.animalstudyregistry.org), set up by the German Federal Institute for Risk Assessment (BfR) to improve the transparency and reproducibility of such experiments, allows scientists to register or ideally to preregister their experiments along with a detailed statistical analysis plan including the definition of endpoints, inclusion and exclusion criteria and sample size calculation. It also facilitates reporting on ARRIVE guidelines, including essential information, such a genetic background, species differences and housing conditions, as well as more detailed modifiers including analgesics or environmental enrichment.

Animal hospitals

The purpose of in vivo experiments is obviously to embrace complexity. Yet, most research animals are still regarded as a complex test tube and experiments are often done in a largely hypothesis‐driven, reductionist way—as a result, it is often hard to see the wood from the trees and important unexpected phenotypes are missed. A more systematic, holistic phenotypic assessment is necessary to uncover unexpected organismal consequences of a genetic perturbation. Often, the skills are missing to allow such a systematic assessment, however: biologists in basic research labs need to work more closely with veterinarians. This approach has advanced in cancer research with the concept of animal hospitals. It is most powerful if in vivo animal preclinical and early clinical studies are aligned and synchronized. For example, drugs can be tested in multiple cancer subtypes in the mouse, to minimize the cost and time required to study responses in human patients 2.

Learning from the clinic

In medicine, extensive regulation has helped to improve research effectiveness dramatically in a number of ways. Blinding and randomization are now standard—compare this to the surprised look from some molecular biologists when this is suggested for cell culture experiments. Pre‐planning statistical analysis is as commonplace as is pre‐registration of full study protocols for clinical trials on platforms such as http://www.clinicaltrials.gov. Generally, pre‐registration of protocols and statistical approaches, with defined endpoints as well as inclusion and exclusion criteria, is a powerful tool to enhance the efficacy of research despite the extra burden. Initiatives such as the Center for Open Science (COS) (https://cos.io) have argued for some years to extend these concepts to all biological research 3. While in principle this is correct, discovery science on complex systems must remain sufficiently nimble to allow adaptation as part of an iterative experimental process, although care must be taken not to start on the slippery slope of HARKing (hypothesis after results). Nonetheless, experiments involving vertebrates can and should be considered more in light of medical research practices than molecular and cell biology practices.

EMBO Press journals will therefore encourage posting of all animal data related to a research paper on the BfR's Animal Studies Registry or other, equivalent repositories.

Pre‐registration encourages improvement of experimentation through community feedback, as well as affording authors with a means to document\xF6and to claim credit for—an experiment in a citable manner early on, analogous to preprints. Note that the registry allows embargoed release to accommodate scooping concerns. Lets align vertebrate and, where possible, all animal experimentation with the valuable lessons learned from clinical research to optimize their efficacy, while maximizing animal welfare. Register your next animal experiment.

EMBO Reports (2020) 21: e49868