Surveying the literature from animal experiments

Editor—We assessed the methodological quality of reviews of animal studies identified using a carefully designed search term combination in Medline and Embase (1996-2004). We also examined bibliographies of known reviews and contacted experts. In total, 30 reviews summarised studies in live animals, which measured laboratory variables or examined treatment effects, identified from search of a publicly available resource. These reviews often lacked methodological features (figure), which increased the risk of biased inferences. We therefore concur with Lemon and Dunnett, that better methods of surveying the literature on animal experiments are needed.¹

Figure 1.

Methodological features of reviews of animal research (1996-2004). Data presented as 100% bars with numbers of studies

However, we do not understand their objection to the use of systematic reviews. Their argument seems to be that it is difficult to combine data from different studies in animal research. Perhaps it needs to be emphasised that meta-analysis is not the objective of a systematic review. The proportion of reviews of animal research that would benefit from meta-analytic techniques is unknown. Among the reviews of animal studies we assessed, 12/30 (40%) used a statistical combination of individual results. The reviews usually did not assess key features in assessing suitability of combining results statistically—for example, exploration of heterogeneity between studies, assessment of study validity, and the risk of missing studies.

These deficiencies interfere with gauging the strength of any inferences. Reviews earn the adjective systematic by use of explicit methods to minimise bias at every step of the reviewing process including literature search, study selection, critical appraisal, and data synthesis. The need for rigour when reviewing animal research is undeniable and systematic review method² provides a sound strategy for conducting such reviews.