Surveying the literature from animal experiments

Editor—We agree with Lemon and Dunnett that better methods of surveying the literature on animal experiments are needed,¹ but we do not share their confidence in the utility of non-systematic reviews.

In clinical trials, systematic review and meta-analysis have made important contributions to our understanding of sources of bias, and the quality of clinical trials has improved as a result. We believe that the same approach can be used to increase our understanding of sources of bias in animal experiments, again leading to improvements in study quality.

Not to publish negative results from clinical trials is widely accepted as unethical, because this may lead to the study being unwittingly repeated by other investigators, exposing trial participants to risk of drug side effects with no prospect of benefit. We consider that non-publication of data from animal studies is equally unethical.

It is argued that tight control of experimental conditions minimises variance and allows for small sample sizes, but most animal studies are still hopelessly underpowered. Systematic review has allowed the analysis of sample size in studies of FK506 in animal models of stroke; the observed variance suggests that 65 animals per group would be needed to give an 80% chance of detecting an improvement in outcome of 20%.² In fact, the largest study reported 16 animals per group and the average was eight animals per group. Performing underpowered studies is as unethical in animal studies as it is in human studies.

Whatever the merits of animal experiments, for many diseases, including stroke, the benefits seen in animal models have been lost in translation. Finding out the reasons for this discrepancy is a matter of some urgency, and a problem to which all available tools—including systematic review—should be brought to bear.