Implementing guidelines on reporting research using animals (ARRIVE etc.): new requirements for publication in BJP

Abstract

The ARRIVE guidelines have been implemented in BJP for 4 years with the aim of increasing transparency in reporting experiments involving animals. BJP has assessed our success in implementing them and concluded that we could do better. This editorial discusses the issues and explains how we are changing our requirements for authors to report their findings in experiments involving animals. This is one of a series of editorials discussing updates to the BJP Instructions to Authors

Video

To view the video on the ARRIVE guidelines, visit: https://www.youtube.com/watch?v=DYXoUAnhoPM

Introduction

As a journal editor, I was invited by the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs) to join the expert working group that helped evolve the ARRIVE guidelines (Kilkenny et al., 2010). So, I have been able to observe the process from the start: all the way from the discussions in 2009 that led to them being written, through their publication in journals in 2010, then the attempt to persuade authors to follow the recommendations, and finally the current stage in 2013–14 of reviewing how successful that has been. Recent evidence suggests that success, generally, has been limited (Baker et al, 2013). This editorial discusses what is needed and lists how BJP is going to deal with it from now on.

Why ‘Transparency’ The prime responsibility of Journal Editors is to publish original science that increases knowledge. For us there is no debate about ‘Transparency’, i.e. making it clear what was done. Methods must permit independent repetition of experiments as a basis for further work. It must be crystal-clear so that other scientists can start where they left off and build on the new knowledge. Editors must also initiate a robust review process, often demanding more detail than the authors thought was necessary, so that readers can be convinced that the process leading to the new knowledge is valid.

Optimising numbers of experiments Confidence in a study avoids the need for many people to repeat experiments as a basis for further work. So it is the responsibility of the Editor to ensure that articles that they publish have the most appropriate number of experiments rather than the smallest number. Underpowered studies are useless. In the long-run studies with an insufficient number of experiments provide less secure results and are more likely to lead to an eventual excess in the number of animals used.

Animal welfare Similarly good animal welfare is a given. Consistency of experiments demands this. Healthy animals are more likely to be like other healthy animals. However, different lots of unhealthy animals are likely to be unlike differently unhealthy animals.

How we implemented ARRIVE

It was decided to publish the guidelines in a range of biomedical journals so as to reach a wider audience and persuade authors that it was relevant for ‘their’ -ology. Therefore, it was published simultaneously in seven journals including BJP.

Kilkenny, C., Browne, W., Cuthill, I. C., Emerson, M. and Altman, D. G. (2010), Animal research: Reporting in vivo experiments: The ARRIVE guidelines. British Journal of Pharmacology, 160: 1577–1579. http://onlinelibrary.wiley.com/doi/10.1111/j.1476-5381.2010.00872.x/full

Specific for Pharmacology

It seemed to us from the start that one size would not fit all but that this generic set of guidelines could be tailored for different subject disciplines. So we formed a group of pharmacology-minded colleagues and when we published the ARRIVE Guidelines in BJP this was accompanied by a ‘Pharmacology-specific’ Editorial to emphasise what we thought was important, and perhaps de-emphasise what was not, for the type of papers that we publish.

McGrath, J., Drummond, G., McLachlan, E., Kilkenny, C. and Wainwright, C. (2010), Guidelines for reporting experiments involving animals: the ARRIVE guidelines. British Journal of Pharmacology, 160: 1573–1576. http://onlinelibrary.wiley.com/doi/10.1111/j.1476-5381.2010.00873.x/full

Pharmacology is the study of how drugs work. The work that we publish is almost entirely based on animals or tissues or cells derived from animals, with some human work where possible. Most of the work concerns identifying drug targets and the mechanisms involved. This is almost exclusively worked up using animals due to the availability and consistency of the material, and the ideas generated are taken to humans once there is a thorough enough understanding of the actions of the drugs for it to be safe and likely to produce some beneficial effect (McGrath, McLachlan & Curtis, 2014)

Implementation of ARRIVE

Like many other journals we then set about recommending ARRIVE to our authors in our published ‘Author Guidelines’. During the review process, many authors evidently paid no attention to ARRIVE and its accompanying editorial, so before accepting their article we reminded them and asked them to take note. We had interesting correspondence with some recalcitrant authors. Some did not see the point, for their work, of some of the guidelines (and they were not always wrong on a detailed level). Others resisted in various interesting ways such as saying that ARRIVE was not relevant because their work was entirely on animal tissue rather than in vivo (and technically they were right since that is where the ‘… IV …’ of ARRIVE comes from; but in my view they were wrong because animal welfare and the three Rs should apply here as well; in vitro is, after all, use of animals in provision of animal tissues.).

Accompanying measures: Statistics and Animal Models

We did not stop there. Over the same period our journal, BJP, launched two Virtual Themed Issues (articles published at different times but brought together as a collection on the internet) that we thought might help to enhance the quality of publication in our subject.

First was ‘Animal Models in Pharmacology Research’, which brought together an irregular series of articles on models for different areas of the subject providing a gold standard for good practice, http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1476-5381/homepage/animal_models.htm.

Secondly ‘Statistical Reporting’ was a series of articles, simultaneously published in sister journals, that attempted to draw attention to important aspects of statistical reporting in an accessible way that biomedical scientists would want to read and learn from. Although excellent statistical guidance is available for clinical research, basic research requires quite different statistics and best practice varies with subject area (Drummond et al., 2011) http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1476-5381/homepage/statistical_reporting.htm.

Have our efforts made a difference?

After running like this for two years we had stimulated some interest evidenced both from ‘discussions’ with colleagues, many of whom remained of the view that this was all a waste of time and effort and, conversely, those advocates of ARRIVE who pointed out that some authors were simply ticking the box for ARRIVE-compliance but were not fully complying.

So we reviewed a random sample of published papers with an eye to whether we were succeeding or not.

The bad news was that we still had deficiencies in some design and analysis matters despite this being a focus of ours; this related largely to clarity of numbers of animals used, often glossing over group sizes that might be too small, leading to exactly the confidence issue we mentioned above. There were also inadequacies in explaining welfare issues in some papers; no suggestion of poor welfare, just lack of detail, again leading to potential loss of confidence for those who might wish to build on the work.

The good news was that authors referred to the ARRIVE guidelines and many made an effort to incorporate aspects that they would not previously have thought to include.

So how did we get it only partly right? The answer lies in the difficulty of managing change in a well-worn process. Everyone bears some responsibility. The authors will resist ‘rules’ that appear to add to the information that they need to provide and, when asked to tick a box to say that they have read and adhered to the ARRIVE guidelines, will happily tick it, most probably without ever having read its twenty sections. Reviewers, who are themselves authors elsewhere, will often take a similar view, and may, in some cases, lack appropriate expertise due to specialisation. This is exacerbated by the reduction of exposure to animal studies at all levels of the education system in response to decades of lobbying against it (so a key objective for the future should be more, not less, education in the techniques of animal research). The editor making the decision to accept or reject a paper will then often be guided by the reviewers and, since the number of ARRIVE-derived aspects will likely have increased beyond the previously normal level, may well be quite happy.

Some journals have sought to stiffen the system by employing a checklist prepared by the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), requiring the authors to not only tick the box but state where in the manuscript they have dealt with the question in hand. This may be better than the dreaded tick-box but it is at the same time cumbersome, in having so many questions, and imprecise, in that some of the questions need a great deal more attention than others. Undoubtedly it is difficult to police. It is also heavily biased towards experiments on animals, although many of the presentational and experimental design issues are generic, applying to almost all studies published in a biomedical journal, whereas animals, particularly in vivo, may be involved only in a proportion.

Next steps

It is too late by the article submission stage to teach anyone about experimental design http://youtu.be/-D0XQ4KQeFM but, by sending out signals about what are no longer acceptable standards, editors can contribute along with educationalists and research funders to raising the bar for acceptable experimental design.

BJP is following these principles of requiring better reporting of experimental design in other research techniques as well as Research involving Animals (Curtis et al., 2015 – Editorial 3).

We have now devised a new aspect of the submission system for our journal that will rely on asking direct questions rather than having only tick boxes. This will be tailored to what we see as being necessary for pharmacological studies, incorporating some ARRIVE guidelines and some of the principles of the Basel Declaration Society, http://www.basel-declaration.org, who are seeking to internationalise good practice (see also McGrath et al., 2015)

Practicalities of collecting and reviewing necessary information

Authors will complete a checklist, ‘Compliance with ethical requirements for studies using animals’, indicating that they have provided necessary information and will be required to provide this information in a template of the Methods Section of the manuscript. This should facilitate its discovery by reviewers and readers and contribute to transparency. The rationale behind this is that the exercise of recommending the inclusion of certain information is pointless unless there is a simple way of the authors providing it, the reviewers finding it to assess it and the readers finding it to evaluate the work. The information must be in the paper and not simply in the journal's review files.

To ensure that this is properly ‘policed’ we will introduce a new and additional aspect of peer-review, involving a team of specialists whose role will be to review those aspects specific to studies involving animals.

The main information to be captured is:

• more robust detail on experimental design on experiments involving animals,

• justification for the choice of experimental model,

• clearer information on the legal and ethical framework and approval by Ethics Committees

• more detail on animal welfare.

Much of this concerns good experimental design and appropriate statistical analysis. It is a common mistake for example, to think that fewer experiments necessarily produce better science. One of the most common errors that we find in review is that too few experiments lead to under-powered studies that do not lead to sound conclusions and so would require more experiments in the long run to clarify false leads.

The checklist on ‘Compliance with ethical requirements for studies using animals’ is part of a larger checklist on Experimental Design, which contains a generic section on 'Compliance with Experimental Design and Statistical Analysis requirements': this generic section will be discussed and included in a separate editorial (Curtis et al., 2015).

Checklist on compliance with requirements for reporting experiments involving animals or animal tissue is in Table 1.

Table 1.

Checklist on compliance with requirements for reporting experiments involving animals or animal tissue

The Checklist below has been developed to guide authors with respect to the nature and level of information that BJP expects in manuscripts that describe experiments that involve the use of animals.

Good science and good animal welfare standards go hand-in-hand and these requirements will assist in improving dissemination of good practice.The rationale is to encourage full disclosure of relevant information that would allow the study to be independently replicated and to enable editors and reviewers to judge whether the ethical quality of the work adheres to the standards that the journal requires (see Author Guidelines; British Journal of Pharmacology, 2015).

If the author is using a well-established and previously described methodology, much of the information can be communicated by reference to published work, but it is essential that the necessary detail is present in the work cited, i.e. don't cite a paper that in turn cites another and so on; cite only the original source. If the methodology, has been modified, this needs to be explained.

If a novel methodology has been developed by the author and shown to have the potential to replace animals, to reduce the number of animals required or to reduce the severity of the procedure (refinement), this should also be explained.

Note that the appropriate number of animals used needs to satisfy experimental design and statistical power requirements (see generic checklist ‘Compliance with Experimental Design and Statistical Analysis requirements’, discussed in a separate editorial (Curtis et al., 2015). An underpowered study is wasteful of animals since it does not provide an answer to the questions posed and must be repeated using more animals. Standard equipment does not need to be described unless a specific model is essential for the study. Equally, standard animal housing and care does not need to be explained in detail as long as these meet the standards required by relevant local guidance or law. Authors will be aware what aspects of their technical approach are important to the quality of their data; these details need to be included.

Some information is mandatory: Details about any legislation or guidance that the work is covered by (including any ethical approval that has been obtained); details about the animals that have been used (e.g. source, species, strain, age, sex); details of what happens to the animals (e.g. drug administration, provision of analgesia or anesthesia, any welfare assessment protocols and humane endpoints, method of killing). Additionally, it should be possible to trace what happens to all animals used in the study (i.e. the numbers need to add up).

A template is provided for inserting this information. The task should not be overly onerous or take up too much space in the manuscript. It will not be included in the word count if it causes the total to exceed the maximum allowed. Instructions will be found in the Author Guidelines (British Journal of Pharmacology, 2015)

Please include in the Methods of your paper a section called ‘Compliance with requirements for studies using animals’. This is provided in the template. Include all items and if any are ‘not applicable’ explain this briefly.

Full disclosure is required1.

Methods Section:

Validity Provide a scientific justification for the animal species and model selected for study, making reference to relevance to human biology/therapeutics if appropriate..

Ethical statement Indicate the nature of the ethical review permissions, relevant legislation (e.g. Animals [Scientific Procedures] Act 1986), and national or institutional guidelines for the care and use of animals, that cover the research.

Animals Provide details of source, species, strain, sex, range of age and weight of animals and any additional data that are relevant to the study, for example genetic status, health status, any re-use or continued-use, etc.

Experimental procedures Provide precise details of all experimental procedures (e.g. drug formulation, dose, site and route of administration, anaesthesia and analgesia used, surgical procedures including asepsis, and method of killing). Provide details of any specialist equipment used, including supplier(s).

Housing and husbandry Provide details of: Housing (type of facility e.g. specific pathogen free [SPF]; type of cage or housing; bedding material; number of cage companions; tank shape and material etc. for fish). Husbandry conditions (e.g. breeding programme, light/dark cycle, temperature, quality of water etc., for fish, type of food, access to food and water, environmental enrichment). Welfare-related assessments, measurements and interventions (e.g. humane end points) that were carried out prior to, during, or after the experiment.

Numbers analysed The ‘Experimental Design and Statistical Analysis’ section requires reporting the number of animals in each group included for analysis: if any animals or data were not included in the analysis, explain why.

Discussion Section: include these aspects here

Interpretation & scientific implications Describe any novel aspects of your experimental methods and/or experimental findings that may have implications for the replacement, refinement or reduction (the 3Rs) of the use of animals in research.

Generalisability/translation Comment on whether, and how, the findings of this study are likely to translate to clinical relevance.

¹Disclosure in this context is for transparency, to allow others to repeat the study exactly, to demonstrate that globally recognized norms of animal ethical treatment are followed, and should be sufficiently detailed to allow peer review to verify compliance. Much of the information can be communicated by reference to published work, but it is essential that the necessary detail is present in the work cited, i.e. don't cite a paper that in turn cites another and so on.