Improving the predictive value of interventional animal models data

Abstract

For many chronic diseases, translational success using the animal model paradigm has reached an impasse. Using Alzheimer’s disease as an example, this review employs a networks-based method to assess repeatability of outcomes across species, by intervention and mechanism. Over 75% of animal studies reported an improved outcome. Strain background was a significant potential confounder. Five percent of interventions had been tested across animals and humans, or examined across three or more animal models. Positive outcomes across species emerged for donepezil, memantine and exercise. Repeatable positive outcomes in animals were identified for the amyloid hypothesis and three additional mechanisms. This approach supports in silico reduction of positive outcomes bias in animal studies.

Introduction

The translation of basic biomedical knowledge into effective treatment for human disease has relied heavily on the use of animals as models. However, for many complex disorders, therapeutic success in animals has not been accompanied by similar success in humans [1–3]. One increasingly cited reason lies in flawed animal study design [4–7] and insufficient reporting of methods [4]. These methodology issues probably contribute to failure of repeatability and efficacy within and across the animal–human boundary [8–10].

The reasons for poor translatability of animal data are multifactorial [1]. Apart from demonstrating efficacy in a model system, pharmacokinetic and pharmacodynamic (PK/PD) variables that define the target exposure–response relationship for a given intervention (see Glossary) across species must be established [11]. Recent methodology developments in PK/PD modeling [11,12], pathway-based toxicology [13,14] and emerging multiorgan and 3D tissue culture technologies [15] have the dual benefit of improving predictability of these processes and reducing animal use [16]. Regardless of improvements in these areas, efficacy must still be established – this is likely to continue to drive animal model use for the foreseeable future, despite suggestions that the animal model paradigm is broken [17]. Published literature represents a valuable source of efficacy data, with the caveat that there are some challenges to its interpretation. The current research landscape is one in which the majority of interventional animal studies report improved outcomes [2,19–22]. This trend derives from the combination of reluctance to report negative data [20], and methodology flaws that promote false-positive outcomes [21,22]. One way to assess the therapeutic potential of an intervention is to examine the methodology details of related studies across species and settings [23]. To conduct such a meta-analysis, related studies assessing the same intervention must first be aggregated. For complex diseases driven by multiple intersecting mechanisms [24], it is a challenge to aggregate the evidence for or against the therapeutic potential of a given mechanism or intervention within the full spectrum of current work on the topic. This is further complicated by the increasing number of studies published each year [25], with the attendant risk that many useful animal studies are not identified. This review approaches this challenge by assessing repeatability of interventional outcomes across mechanisms and species, using a networks-based systems approach. For this proof-of-concept review, Alzheimer’s disease (AD) is used as an illustrative example of a complex condition.

Methods

Across 752 animal and human studies and human clinical trials 448 interventions were examined. Briefly, source data were aggregated from PubMed, Mouse Genome Informatics (MGI) and ClinicalTrials.gov using search terms and time limits listed (see supplementary material Table S1 online). Only intervention studies were included, reducing the number of studies to a total of 353 and 148 interventional animal and human studies, respectively, and 251 human clinical trials (see supplementary material Table S1 online). Intervention studies were defined as those in which the effect of a broadly defined intervention (pharmaceutical, phytochemical, physical, genetic, behavioral or environmental) on the AD phenotype was examined. Next, the following information was collected for each clinical trial or study: patient population (human studies), model (for animal studies), intervention, outcome and mechanism. Outcomes described the effect of an intervention on overall disease severity in a patient population or animal model system. In the majority of human studies, outcomes were based on clinical measures, whereas in animal studies a range of biomarkers, pathologic scores and/or functional measures were used. Each study was also assigned a mechanism based on the mechanism of action of its intervention. To establish a controlled vocabulary across species, mechanisms were defined by the gene ontology project terms. Related interventions were aggregated within their cognate mechanisms, thus providing a central framework around which to organize interventional studies. Individual data segments (patient population, model, intervention, outcome and mechanism) were arranged in binary form, to denote whether two entities (nodes) had a relationship (edge) or not, without imposing a preferential value on any relationship. To visualize the network, a freely available network program, Biolayout Express 3D 3.0 (http://www.biolayout.org/) was used. Detailed methods used to aggregate the dataset for the review (Table S1), lists of search terms (Table S2), results of searches (Tables S3–S6; Figures S1,S2) are presented in the supplementary material.

Results

Utility of the network

The simplest use of the network is rapidly compiled animal and human outcomes data around a given intervention (supplementary material Figure S1), as well as to identify studies across related interventions that map to a common mechanism (Figure S2; Table S6a,b). Additionally, for the purposes of this review, the network was interrogated to explore patterns regarding translatability of animal model data in AD. Four questions were posed: (i) can patterns of animal model use be identified? (ii) How do outcomes segregate across species? (iii) What mechanisms have been studied across species? (iv) Can interventions and mechanisms with translational potential be identified? For the purposes of this review, interventions with translational potential were defined as those in which evidence was provided for similar outcomes across multiple species and settings.

Can patterns of animal model use be identified?

Several findings emerged that are likely to contribute to failure of promising preclinical candidates to translate in human AD trials. Of the 139 models across 11 species included in the network only 20 were used in four or more studies, and one model [Tg(APPswe, PSEN1dE9)85Dbo] was used in 24% of studies (Table 1). This is likely to represent a form of publication bias, in which well-accepted models engender continued use. The most heavily used strains, with the human AD alleles they model, and their strain background (as designated by MGI, see supplementary material online) are listed in Table 1.

Table 1.

Most heavily used animal models in the network

Animal model	Studies (%)	Human allele(s)a	Strain backgrounda
APP/PS1:Tg(APPswe,PSEN1dE9) 85Dbo. PMID: 11337275	85 (24%)	APP KM595/596NL Swedish PSEN1 deltaE9	(C57BL/6 x C3H)F2
APP695swe: Tg(APPSWE)2576Kha. PMID: 8810256	42 (12%)	APP K670N/M671L Swedish	(C57BL/6 x SJL)F1
3xTg-AD: B6.Cg-Psen1tm1Mpm Tg(APPSwe,tauP301L)1Lfa. PMID: 12895417	38 (11%)	APP K670N/M671L Swedish PSEN1 MAPT P301L	C57BL/6J
5XFAD: Tg(APPSwFlLon,PSEN1*M14 6L*L286V)6799Vas. PMID: 17021169	19 (5%)	APP K670N/M671L Swedish APP V717I London APP I716V Florida PSEN1 M146L PSEN1 L286V	(C57BL/6 x SJL)F1
Rodent: cerebral injection of amyloid beta oligomers. PMID: 15810904	17 (5%)	None	Inbred/outbred strains
APPPS1-21:Tg(Thy1-APPSw,Thy1-PSEN1*L166P)21Jckr. PMID: 16906128	14 (4%)	APP K670N/M671L Swedish PSEN1 L166P	C57BL/6J
CRND8: Tg(PRNP-APPSweInd)8Dwst. PMID: 11279122	14 (4%)	APP K670N/M671L Swedish APP V717F Indiana	(C3H/HeJ x C57BL/6J)F1
PDAPP-J20: Tg(PDGFB-APPSwInd)20Lms. PMID: 10818140	13 (4%)	APP K670N/M671L Swedish APP V717F Indiana	(C57BL/6 x DBA/2)F2
TASTPM: Tg(APPSWE)2576Kha/Tg(PDGF B-PSEN1M146L). PMID: 15261108	9 (3%)	APP K670N/M671L Swedish PSEN1 M146L	(C57BL/6 x SJL)F1SW x (C57BL/6 x DBA/2)F1
APP23: Tg(Thy1-APP)3Somm. PMID: 9371838	8 (2%)	APP K670N/M671L Swedish	(C57BL/6J x DBA/2)F1
SAMP8 mice. PMID: 24269312	8 (2%)	None	AKR/J
PDAPP: Tg(APPV717F)109Ili. PMID: 7845465	6 (2%)	APP V717F Indiana	Not specified
APP/PS1: Tg(Tg(Thy1-APPLon)2Vln,Thy1-PSEN1*A246E)2Vln. PMID: 10964951	5 (1%)	APP V717I London PSEN1 A246E	FVB/N
APP751: Tg(Thy1-APPSwDutLon)101Lpr. PMID: 14637096	5 (1%)	APP KM595/596NL Swedish APP E618Q Dutch APP V6421 London	CBA x C57BL/6
Tg(Thy1-APPSwDutIowa)Bwevn. PMID: 14985348	5 (1%)	APP E693Q/D694N vasculotropic Dutch/Iowa APP K670N/M671L Swedish	C57BL/6
APP/PS1: B6C3H-Tg (APP695) 3Dbo Tg (PSEN1 A246E) 5Dbo/J; PMID: 9354339	4 (1%)	APP KM595/596NL Swedish PSEN1 A246E	(C57BL/6J x C3H/HeJ)F2

Each model is defined by three identifiers: a synonym, the precise strain nomenclature and the PMID of the seminal manuscript describing it.

^aData from Mouse Genome Informatics (MGI; http://www.informatics.jax.org)

Strains carrying the rd-1 mutation (red): ABJ/LeJ; BDP/J; BUB/BinJ; C3H and all substrains; CBA/J; CBA/N; FVB/NJ; JGBF/LeJ; MOLD/RkJ; MOLF/EiJ; NFS/N; NON/LtJ; P/J; PL/J; RSV/LeJ; SB/LeJ; SF/CamEi; SF/CamRk; SK/CamEi; ST/bJ; SJL/J; SWR/J; WB/ReJ; WC/ReJ (http://eyemutant.jax.org/rd1.html).

Precise definition of rodent models was rarely provided. Mouse models were often identified using synonyms (e.g. the synonym APP/PS1 denotes four different models; Table 1). Model nomenclature defining the mutant allele was usually provided in the methods or references; however background strain was rarely described. Because precise strain nomenclature was incorporated in the dataset, the presence of potential genetic confounders across all genetically altered mouse studies could be identified. Of the 16 strains in Table 1 (comprising approximately 80% of animal studies in the network), eight contain contributions from strains carrying the rd-1 allele. This autosomal recessive allele results in photoreceptor degeneration and renders animals blind by 6–8 weeks of age [26]. Because its presence cannot be identified clinically, the rd-1 allele is propagated in models on mixed backgrounds, thus rendering an unknown proportion of mutant and control populations blind. Inconsistent results and high within-group variability of mutant and wild-type groups in spatial tests used to detect memory impairment have been noted [27–29]. One commonly used test, the Morris water maze, has been shown to be strongly impacted by presence of the rd-1 mutation associated with the SJL background [30] in a commonly used model [Table 1; APP695swe: Tg(APPSWE)2576Kha]. This model is also available on the 129S6 background strain (http://www.taconic.com/2789), which carries the Disc1 mutation known to affect working memory in mice [31].

A further three models in Table 1 contain contributions from the DBA/2 strain. DBA/2J mice harbor two mutations that result in progressive eye abnormalities that closely mimic human hereditary glaucoma [32]. In total, 55% of the interventional studies using AD models in 2013 were done in populations potentially carrying mutations (at unknown frequencies) that impair vision. The potential impact of confounding alleles is compounded by the tendency for animal studies to use small sample sizes [10,33], resulting in overestimation of effect size, high false-positive rates [7] and poor reproducibility. Studies could indicate that the parent strain has been backcrossed to a single strain, usually C57BL/6, and not all studies employ cognition testing that could be impacted by blindness. However, if this is not noted, or the use of F1 hybrids (without subsequent breeding) between rd-1 and non-rd-1 carrying strains is not explicitly stated, the distribution of blinding alleles in mutant and control populations cannot be accurately assessed.

Third, variations in species physiology that impact translatability were identified. These included descriptions of strain variation in processing of amyloid-beta [34], altered phenotype expression by strain [35–37], variations in behavioral testing by strain [38,39], species similarities [40] or differences in molecular signatures of the aging synapse [41], or aging brain [42,43], gender differences in amyloid plaque development within model [43,44] and differences in amyloid-beta degradation and toxicity between mice, humans and large animal models [45,46]. This type of normative data profoundly impacts study design and interpretation, and contributes to the species barrier [18]. Unfortunately, it is difficult to find without prior knowledge of what to look for.

How do outcomes segregate across species?

Interventional outcomes across humans and animal studies revealed a translational gap (Figure 1; Table 2). The majority of animal studies in 2013 (75.9%) reported an improved outcome. Just under half of published human studies (46.6%) over a five year period reported improvement. It should be noted that the denominator of the human data included published reports only (Table 2), and thus excluded the majority of human clinical trials. The majority of human clinical trials (89%) have no results available through the ClinicalTrials.gov site. Animal studies with a worsened outcome (13.8%) were predominantly those where the intent was to worsen some aspect of the AD-like phenotype (e.g. cholinergic denervation) [47]. Therefore, these are not directly comparable with the few human studies in which an intervention intended to treat the disease actually worsened outcome [48–51].

Figure 1.

Interventional outcomes across humans and animal studies. (a) Combined human and animal network. The majority of human clinical trials have no available results (yellow nodes). The majority of animal studies (blue nodes, 75.9%) reported an improved outcome. (b,c) Individual outcomes networks for humans (b) or animals (c). Just under half of human studies (b, blue nodes; 46.6%) over a five year period report improvement. Animal studies with a worsened outcome (c, red nodes; 13.6%) were predominantly studies where the intent was to worsen some aspect of the AD-like phenotype. Therefore, these are not directly comparable with the few human studies in which an intervention intended to treat the disease actually worsened outcome (b, red nodes; 2.7%). Interventions producing no effect constitute over a quarter of human studies over five years (b, orange nodes; 26.4%) but only a small percentage of animal studies report no effect (c, orange nodes; 4.8%).

Table 2.

Outcomes of interventional studies for Alzheimer’s disease (AD) across species

	Animal studies (2013)	Human studies (2013)	Human studies (2009–2013)
Improved	268 (75.9%)	24 (57.1%)	69 (46.6%)
Mixed outcome	22 (6.2%)	4 (9.5%)	13 (8.8%)
No effect	14 (3.9%)	7 (16.6%)	39 (26.4%)
No adverse effect	0	6 (14.3%)	23 (15.5%)
Worsened	49 (13.8%)	1 (2.4%)	4 (2.7%)
Total	353 (100%)	42 (100%)	148 (100%)

What mechanisms have been studied across species?

Animal and human studies mapped to a total of 81 different mechanisms (supplementary material Table S3), implicating a process of cumulative decline across multiple cellular systems in AD. Of these, 42 mechanisms were shared across humans and animals (i.e. they contained at least one intervention tested in humans and animals. The proportion of studies assigned to these common mechanisms differed across species (Figure 2; supplementary material Table S3). Studies targeting neurotransmitter systems predominated in humans (constituting 38.8% of all interventions in humans compared with 10% in animals). This reflects interventions across multiple neurotransmitter systems for psychological manifestations of AD such as apathy, depression, aggression and agitation [52–54]. Within this category, studies targeting the cholinergic system (GO:0015464 – acetylcholine receptor activity) predominated in humans and animals. Interventions that damage the cholinergic system in mice [47,55] and monkeys [56,57] promote amyloidogenic damage and impair cognition, thus providing evidence for intersecting cholinergic and amyloid hypotheses. Studies focused on amyloid-beta-related interventions were similarly distributed across species (21.8% and 18.5% of interventions in humans and animals, respectively; blue shading in Figure 2). Studies examining mechanisms that can be ascribed to lifestyle changes of diet, exercise or neutraceutical supplementation (GO:0042593 – glucose homeostasis; GO:0014823 – response to exercise; GO:0006766 – vitamin metabolic process; GO:0007584 – response to nutrients) assume almost identical proportions across humans and animals (11.2 and 11.6%, respectively). Sixty-four percent of 25 human studies across these four mechanisms demonstrate improved outcomes. Of the 30 studies examining the outcomes of three approved interventions that map to cholinergic mechanisms (GO:0015464 – acetylcholine receptor activity – donepezil, rivastigmine and galantamine), 20 (66%) had improved outcomes. Interventions rooted in inflammation (red box in Figure 2), oxidative stress (gray box in Figure 2) and glucose homeostasis (purple box in Figure 2) prevailed in animals and reflect the relatively greater ease with which biomarkers for these mechanisms can be quantified in the brains of animals compared with humans.

Figure 2.

Mechanisms of disease in AD across humans and animals. The first 16 mechanisms shared across humans and animals (supplementary material Table S5) are compared using a stacked 100% bar. Studies targeting neurotransmitter systems are indicated by a thin horizontal black bar and constitute the first eight mechanisms in Table S5. These predominate in humans compared with animals. Within this category, studies targeting the cholinergic system (bright orange; GO:0015464 – acetylcholine receptor activity) predominate in humans and animals. Studies focused on beta-amyloid-related interventions were similarly distributed across species (blue shades, thick horizontal grey bar). Interventions in which outcomes describe inflammation (gray), oxidative stress (red) and glucose homeostasis (purple) prevail in animals. Full data are given in Table S5.

Can interventions and mechanisms with translational potential be identified?

The large number of positive outcomes in animal studies presents an interpretive challenge. To address this, a three-step approach was used. First, (i) the network was interrogated to identify characteristics of animal studies for those interventions of known benefit in humans. Next, (ii) the repeatability of outcomes for those interventions that had been examined across multiple animal models or species was assessed. Last, (iii) the repeatability of outcomes across multiple animal models or species for those mechanisms harboring related interventions was assessed.

• What are the characteristics of animal studies for interventions known to improve outcomes in humans? Twenty-five interventions shared across humans and animals were identified (supplementary material Table S4). Two further criteria were applied to these 25 interventions. First, multiple studies had to be available in animals and humans. Second, only those interventions in which outcomes were predominantly improved in human studies were selected. Only three interventions survived these criteria – two approved drugs (donepezil and memantine) [58] and exercise. In all three, two characteristics of the cognate animal studies were noted: outcomes were improved in the majority of animal studies and improved outcomes in animals were distributed across a range of different animal models (three or four distinct models or species). The latter finding is reminiscent of recommendations to synthesize data from multiple animal studies to assess likelihood of translation [10], and is the second requirement of the FDA Animal Rule [59]. In the USA, the Animal Rule is a regulatory means to approve interventions in which efficacy trials in humans are not possible – as such, it provides more-comprehensive guidance regarding efficacy in animals than the traditional Investigational New Drug (IND) exemption [60]. The majority of interventions in animals did not have cognate studies in humans but could still have potential translational significance. To evaluate the repeatability of outcomes across these interventions in animals, the next step was to determine which interventions had been performed across a range of animal models, and what their outcomes were.

• Relatively few interventions have been repeated using different animal models. Eighteen interventions that had been repeated across three or more distinct animal models (supplementary material Table S5) were identified. For each, the network delivered the spectrum of outcomes. Most interventions were associated with a range of outcomes. Further evaluation of the methods of constituent papers was needed to clarify the reasons for this in each case. An example of how intertwined variables rooted in methodology and the species barrier can be seen in studies describing the use of the RXR agonist bexarotene in mice [61–64]. Bexarotene was initially reported to improve cognition and amyloid-beta plaque deposition in three mouse models [61]. Subsequent reports [62–64] were unable to replicate these findings. Potential explanations advanced for this discrepancy included variations in drug formulation [64] and sample sizes that were too small with respect to variability in plaque deposition by age and gender in the model used [62,63,65]. Two interventions in the ‘worsened’ category (streptozotocin and hypoxia) were identified (supplementary material Table S5). Streptozotocin-based interventions across four different animal models support the association between metabolic dysfunction and AD [66], and hypoxic events that worsen the AD-like phenotype support the deleterious role of sleep apnea, stroke or prenatal hypoxia in human AD [67]. As expected, two interventions in the ‘improved’ category were for already approved drugs (donepezil and memantine) [58]. Of the remaining five improved interventions, all were based in the amyloid hypothesis (supplementary material Tables S5,S6).

• Assessing repeatability of mechanistically related interventions. The vast majority of interventions had been examined just once, thus precluding comparison of outcomes across models. However, even though most interventions in animals had been tested only once, several interventions could be mapped to a similar mechanism of action (e.g. phosphodiesterase inhibitors; supplementary material Figure S2). Therefore, the repeatability of outcomes across multiple animal models or species for those mechanisms harboring related interventions was assessed. For this analysis, search criteria were defined as those mechanisms in which related interventions had yielded uniformally improved outcomes in three or more animal models. Five such mechanisms were identified. These, and the related human studies and clinical trials, are given (supplementary material Table S6a). Three novel mechanisms (GO:0004112 – cyclic-nucleotide phosphodiesterase activity; GO:0038180 – nerve growth factor signaling pathway; GO:0048863 – stem cell differentiation) in which all animal studies having improved outcomes were identified. The remaining two were centered in the amyloid hypothesis, which, together with results of the previous search in which amyloid-based interventions emerged (supplementary material Table S5), prompted a final search to aggregate all animal and human data around the amyloid hypothesis in the dataset. This provided an example of how data regarding a given mechanism could be aggregated in an unbiased manner for subsequent methodology analysis to evaluate pharmacologic and other variables that would impact translatability [23] (supplementary material Table S6b).

Concluding remarks

The analysis described in this paper indicates that the proportion of positive outcomes in animals is far higher than in humans for AD-related interventional studies. Given similar conclusions for other diseases [1,2,21], and the dismal therapeutic reality of AD, this is not a surprise. What is surprising, however, is the magnitude of positive outcome data in animal studies (75% in just one year – 2013). The predictive validity of animal data is the subject of increasing scrutiny [2,3,10,17], and animal models are often criticized for failure to provide translational data [18,68]. The reasons for this reflect enmeshed realities of defining target exposure–response relationships across species [11,12], sources of bias alluded to in the introduction [5,6,17,19–22,69] and the species barrier. In response to identified methodology problems, recent guidelines for improved study design [33,70–73] and reporting [69,74] have emerged, and pressure to apply similar design standards to animal studies [75] and clinical trials is growing. If these reforms are consistently upheld by funding agencies and journal reviewers – this is a work in progress [75] – sources of bias that currently reduce credibility of animal studies will be reduced.

This network-based approach identified strain background as a pervasive potential confounder in mouse studies. Full strain nomenclature was used to define each model in the network. This often required manual searching of methods or references, and background strain was rarely defined. Strain assignation to the identified model using MGI revealed the rd-1- and DBA/2-associated blinding alleles, of which the former was noted to confound results of the Morris water maze in a heavily used model of AD [30]. rd-1 is arguably the best known genetic confounder and information regarding prevalence and detection not only of rd-1 but for many other confounders in mice has long been available (http://www.informatics.jax.org/). Nevertheless, awareness of this allele in our dataset appears to be low. Therefore, it is likely that awareness of the many more subtle confounders and species variations such as those we identified in our dataset [29,34,41,43,44,46] will continue to be low. Solutions to this problem are twofold. First, precise model and strain nomenclature should be required by funding agencies and journal reviewers. Second, although confounding alleles are provided in public databases (http://www.informatics.jax.org/), reports describing biological effects of strain-specific confounders on disease mechanisms are buried in the literature. Dissemination of these data would require collation in a comprehensive and user-friendly format that does not rely on prior knowledge to find them.

A second source of variability that, in aggregate, is likely to impact the predictive validity of animal models concerns the relationship between an intervention, engagement of its target, alteration of appropriate biomarkers and altered functional outcome across species. Outcomes measures across humans and animals are difficult to reconcile for AD. Diagnosis of AD in humans rests primarily on clinical criteria [76]. By contrast, most rodent models are defined by genetic and biomarker-driven criteria (Table 1), and cognitive tests such as the Morris water maze are used to describe functional outcomes [81]. If clinical criteria similar to those defining human AD were used to define mouse models, the clinical progression seen in rodent models at the time of intervention would most probably fall into the category of mild cognitive impairment [77,78]. Therefore, as has been noted for ALS [70], it is likely that interventions in mice occur at a comparatively earlier time point than they do in human clinical trials. Additionally, an intervention could engage its target or change a biomarker, but this does not always correlate with altered functional outcome [79]. Demonstrating this chain of events across multiple species or systems increases confidence for translatability. An example of this is animal studies describing proprotein convertase subtilisin/kexin type 9 (PCKS9) inhibition [80,81]. Whereas observing similar outcomes across models is not itself proof of translatability, it does serve as a proxy marker for those mechanisms or interventions that are robust enough to achieve similar outcomes across different systems. This approach has been proposed by others [10], and is the second prerequisite of the Animal Rule [82]. In this review, this principle was combined with a systems biology networks-based method [83].

By combining these concepts, a method to obtain the following translationally useful information was achieved. First, animal and human data around any intervention or mechanism within the dataset could be rapidly compiled. Second, the network could be interrogated to reveal translationally useful patterns regarding animal model choice, outcomes across species, distribution of mechanisms studied across species or the impact of potential genetic confounders on interpretation of animal data. Lastly, by imposing various criteria (in this case, repeatability of outcome across multiple models), the network could be used to identify those mechanisms (within a landscape of many mechanisms) for which sufficient evidence had been collected in animals to allow further analysis for potential translation to humans. In effect, this approach allowed in silico reduction of positive outcomes bias present in animal studies. Once groups of studies have been aggregated by search criteria, the methodology of each must still be scrutinized [23]. The many variables that influence translatability (e.g. PK/PD variables, biomarker validity, time of intervention and termination, selection of outcomes measures across species) are described in the body of each paper, and are best evaluated by investigators, not a search tool.

The failure of animal models to provide translational data is a complex issue. If most animal data cannot be translated, the disconcerting conclusion is that many animal studies are potentially wasted. Tools that can integrate data from extant public databases to reveal translationally useful relationships would improve the utility of available animal model data. Such an overview for any given topic could guide design of related animal studies so as to avoid unnecessary duplication, or address aspects that remain unexamined by existing studies. The approach used to perform this review was developed with semi-automation in mind. All studies falling within given criteria of initial dataset collection were equally represented (i.e. no filter assessing the validity or quality of a study was imposed). Because of this, initial dataset collection was relatively unbiased in comparison with a traditional review. Because outcome, intervention and species or model are usually reported in the abstract of published studies, these represent variables that are accessible and potentially amenable to automated searching. Lastly, data were arranged around a mechanism-based framework using gene ontology, thus allowing the user to relate studies and outcomes to potential therapeutic targets. Semi-automation of this process would allow users to analyze data that would be periodically updated, and cumulative, thus allowing analysis of progressively larger datasets over time. One source estimates that approximately half of the National Institute of Health (NIH) budget is spent on animal research [78]. The animal model paradigm is unlikely to change soon, but methodology advances [18], changing perspectives [10,17] and economic realities [84] suggest that the days of ‘business as usual’ on this front are over.

Highlights.

• 448 interventions across 752 studies for Alzheimer’s disease were compared

• A networks-based systems biology approach was used

• Repeatability of outcomes across species by intervention and mechanism was assessed

• 75% of animal studies reported an improved outcome

• Strain background was a significant potential confounder

• 5% of interventions had been tested across species or across multiple models

• This approach supports in silico reduction of positive outcomes bias in animal studies

Glossary

Intervention

Introduction of variable (pharmaceutical, phytochemical, physical, genetic, behavioral or environmental) into a model system with the intent to assess its effect upon outcome

Mechanism

Means by which an intervention influences a biological system such as a mechanism of action of a drug, effect on a cellular pathway or a physiologic function

Model

An animal system harboring specific AD-related alleles (e.g. APP E618Q), or an animal in which some mechanistic aspect of AD is modeled (e.g. intracerebroventricular streptozotocin injection in wild-type animals)

Outcome

Effect of an intervention on overall disease severity in a patient population or animal model system, as defined by the authors in the abstract

Patient population

These constituted study populations with diagnosis of late-onset Alzheimer’s disease, familial Azheimer’s disease or normal aging, as defined in publications and in ClinicalTrials.gov

Strain background

The rodent strain upon which the mutant allele resides (e.g. C57BL/6J)