Systematic review and meta-analysis of pre-clinical research: the need for reporting guidelines

Abstract

Animal studies should be reviewed more objectively before embarking on human clinical trials, hence the need for guidelines argues the NC3Rs

Animal studies are often used to inform clinical practice, but is the decision to conduct clinical trials supported by reliable laboratory evidence? One would expect that the answer to this would be yes and the decision to test a new treatment in humans to be based on clear evidence of efficacy in animal models. It would also be reasonable to expect pre-clinical evidence to be assessed in a comprehensive and objective manner to ensure an unbiased decision. Two recent systematic reviews—one of animal data¹ and one of human data²—of the effect of calcium channel blockers in stroke have however shown that this is not always the case, with the decision to perform clinical trials based on insufficient evidence.

Systematic reviews and meta-analyses of animal data are still rare compared with the clinical field. Those that have been conducted are usually from the CAMARADES group (the Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies) which have focused on stroke,³ Parkinson’s disease,⁴ and multiple sclerosis.⁵ There are many obstacles to overcome before systematic review and meta-analysis of in vivo data become standard practice, the main ones being publication bias (studies with no significant results are less likely to get published) and the poor reporting quality of animal studies.

The latter is a particular problem. First, it prevents a rigorous quality assessment of the studies to be included in such reviews. While unrandomized studies would not be included in a meta-analysis of clinical trials, using such criteria in a recently published meta-analysis of the effects of anti-emetics in a ferret model of chemotherapy-induced emesis⁶ would have implied the exclusion of every single study! Very often, animal studies fail to report the steps taken to minimize the effects of subjective bias when allocating the animals to treatments or assessing the results for example. This was demonstrated in a survey carried out by the UK Government funded National Centre for the Replacement, Reffnement and Reduction of Animals in Research (NC3Rs). The study reviewed 271 publications of publically funded in vivo research in the UK and USA and found that only 13% of studies reported randomization and 14% of studies using qualitative score reported blinding.⁷ These findings were echoed by other surveys carried out in various fields of research.

Secondly, failure to report features likely to be associated with heterogeneity, such as drug dose, route of administration, sex, and age of the animal used, greatly diminishes the potential of retrospective analyses and the potential to make maximum use of all of the data available. It creates the dangerously false assumption that results can be generalized when they might not be. It also prevents the investigation of sources of heterogeneity, which could reveal whether a treatment has a different efficacy in males and females, for example, or different adverse effects in young and old animals. This information should in turn inform the design of clinical trials or even the decision to start a clinical trial.

The NC3Rs has developed guidelines to improve the reporting of animal studies. The guidelines are called ARRIVE (Animal Research: Reporting In Vivo Experiments; www.nc3rs.org.uk/ARRIVE) and they describe the minimum information that should be included in a manuscript, ensuring that studies are reported in a clear and comprehensive manner, reflecting the study design and conduct. These guidelines are the animal research equivalent to the CONSORT statement,⁸ which has contributed greatly to improving the reporting quality of randomized controlled trials.⁹ The ARRIVE guidelines include advice on describing the background and objectives, methods, results, and interpretation of a study. They also recommend including information related to the ethical approval and funding sources, ensuring that the reviewers and readers are provided with all the information necessary to scrutinize and assess publications accurately. Specifically, the guidelines encourage a detailed description of the study design, a clear definition of the experimental outcomes, and the reporting of results with a measure of precision. They also prescribe a detailed account of the experimental procedures, characteristics of the animals used, and adverse effects. Altogether, this information will provide the means for future systematic reviews to inform the design of clinical trials accurately and improve the translation of animal findings into clinical results.