The promise and reality of therapeutic discovery from large cohorts

Abstract

Technological advances in rapid data acquisition have transformed medical biology into a data mining field, where new data sets are routinely dissected and analyzed by statistical models of ever-increasing complexity. Many hypotheses can be generated and tested within a single large data set, and even small effects can be statistically discriminated from a sea of noise. On the other hand, the development of therapeutic interventions moves at a much slower pace. They are determined from carefully randomized and well-controlled experiments with explicitly stated outcomes as the principal mechanism by which a single hypothesis is tested. In this paradigm, only a small fraction of interventions can be tested, and an even smaller fraction are ultimately deemed therapeutically successful. In this Review, we propose strategies to leverage large-cohort data to inform the selection of targets and the design of randomized trials of novel therapeutics. Ultimately, the incorporation of big data and experimental medicine approaches should aim to reduce the failure rate of clinical trials as well as expedite and lower the cost of drug development.

By simply noting facts, we can never succeed in establishing a science. Pile up facts or observations as we may, we shall be none the wiser.

— Claude Bernard, An Introduction to the Study of Experimental Medicine, 1865

Introduction

We are experiencing unprecedented growth in the amount of biological and medical information collected from human populations. Large prospective cohorts, such as the UK Biobank (1), the All of Us Research Program (2), and the China Kadoorie Biobank (3), are generating increasingly broad and detailed phenotypic descriptions of health trajectories for millions of individuals. Overall, initiatives in more than 30 countries have established more than 60 cohorts, each enrolling at least 100,000 individuals, collectively projected to include as many as 36 million participants (4). For a typical participant, a comprehensive picture of their physical state is provided by fine-grained data collected across various biological domains, including genetics, biomarker profiling, and biomedical imaging. Data from personal electronic devices are harvested continuously to capture physical activity, dietary habits, and social interactions. Streams of biological data are ultimately integrated with medical histories, made available by the rising adoption of electronic health records, to create complex models predictive of medical outcomes (5).

Perhaps not surprisingly, the rapid growth in health and genetic data has led to an explosion in the number of observations connecting physiological traits and diseases to genetic variants that may demark candidate targets for therapeutic intervention (Table 1). This increase in targets has been driven, at least partially, by widespread genome-wide association studies (GWAS) that measure statistical relationships between genetic and phenotypic variation among individuals in a population (6). GWAS and other analytical methods, applied to larger and larger data sets, have uncovered more and more genetic variants with smaller and smaller phenotypic contributions, and have informed our appreciation of the genetic complexity of human disease (6).

Table 1. Summary statistics of known gene-disease associations as reported in various database collections.

However, our ability to uncover genetic disease associations has far outpaced our ability to understand them and, even more so, to act on them. It is abundantly clear that only a small fraction of these associations can be functionally tested, and if we are to use these genetically inspired hypotheses for drug development and clinical testing, the list will need to be prioritized so as to avoid increasing the rates of failure in clinical trials.

Indeed, failures in clinical trials are far more frequent than successes. Among drugs entering clinical development, only about 10% will ultimately pass the stringent regulatory requirements necessary for a new-drug approval (7). The few successful trials bear the expense of all the failed ones, leading to the ever-increasing financial cost of drug development (8).

As the accumulation of human population data and the resulting gene-disease associations continues to increase, a question becomes central to the future of drug development: how can we mitigate the costs and improve on the success rate of clinical trials? Among the many strategies to tackle this question, a fundamental one is to reduce the number of candidate targets before they reach the clinical stage and to enrich them for the most promising hypotheses. Here we highlight four complementary avenues to achieve this goal. First, large and diverse cohorts provide greater power for discovery and fine mapping of likely causal variants, refining potential hypotheses of the effects of genetic variants. Second, the acquisition of intermediate phenotypes, bridging genetic variants and clinical manifestations, enriches our understanding of disease etiology and informs the design of more rational therapeutic strategies. Third, the development of more accurate and interpretable statistical approaches, especially those integrating orthogonal data types, helps prioritize targets and eliminates the least promising ones in silico. Finally, testing candidate interventions using deep and perturbed phenotyping in relatively small studies (i.e., experimental medicine) helps validate hypotheses, refine selection of patients and their appropriate dosages, and reduce the probability of failure at later clinical stages. The ability to move from hypotheses generated in large data sets to validation, integration, and hypothesis testing in small numbers holds the promise of a more efficient approach to drug development.

Genetically inspired target space

Many diseases arise from a complex interplay between genetics, environment, and time-dependent interactions between the two. Although heritability estimates are highly trait specific, most studies report some heritable component (9), suggesting that genetic studies may be useful for understanding pathophysiology and possibly identifying candidate targets for clinical development.

Genetic studies using pedigree- and linkage-based approaches (10, 11) have proved very effective for identifying genetic associations with Mendelian disorders (12). More recently, study designs (13, 14) enabled by inexpensive genotyping have mapped associations between genetic variants and thousands of diseases and quantitative traits (15). Driven by the growing size of sampled human populations and the diversity of measured phenotypic traits, as many as 32,000 gene-disease associations have been mapped so far (Table 1), and many more are expected in the near future.

An overview of all gene-trait connections discovered to date reveals a complex picture (Figure 1A). On one hand, many genes exhibit a high degree of pleiotropy and appear to be associated with many seemingly unrelated traits and diseases (Figure 1B). On the other hand, many traits and diseases are highly polygenic (Figure 1C). The complex gene and protein interactions that likely underlie pleiotropy and polygenicity are such that therapeutically intervening on a single gene-trait link without perturbing other neighboring connections is unlikely. Moreover, intervention on any significant fraction of these connections, aiming to alleviate a reasonable portion of human diseases, also stands as an intractable problem because of the sheer number of clinical trials that would be required.

Figure 1. The polygenic and pleiotropic space of GWAS associations.

The complexity of the gene-trait association network hinders the development of targeted interventions. (A) A representative network derived from the National Human Genome Research Institute (NHGRI)/European Molecular Biology Laboratory–European Bioinformatics Institute (EMBL-EBI) GWAS Catalog shows 6348 associations between 2939 genes and 650 traits (15). (B) Pleiotropic genes show associations with multiple phenotypic traits. (C) Polygenic traits are affected by multiple genes.

Despite recent progress in identifying variant-trait associations, there is rarely a direct path from a statistically significant variant-trait association to a testable therapeutic hypothesis, posing a major challenge for translational applications. Common variants, which drive a substantial portion of the heritability of complex traits (16), can occur in haplotypes, i.e., large chromosomal regions that tend to be inherited together. The non-independent segregation of variants within haplotypes, known as linkage disequilibrium (LD), makes it difficult to identify the causal variant(s) and thereby the mechanisms driving the association. Moreover, disease-associated loci are frequently found in noncoding genomic regions (17). As our current understanding of the regulatory landscape of the genome is incomplete, we often cannot infer directly the underlying gene(s) or other intermediate trait(s) that mediate the observed genetic association. Such knowledge is critical to generate clear hypotheses that are testable in a clinical setting. By prioritizing the collection of intermediate phenotypes — comprehensive molecular and physiological readouts — along with the development of advanced analytical methods that incorporate known regulatory features of the genome, it may nevertheless be possible to identify candidate targets with clear hypotheses that can be validated through carefully designed functional studies and clinical experiments.

Refining and replicating statistical associations

One of the biggest operational challenges in the analysis of large cohorts is replication of findings (18). This is a particularly difficult problem for associations with small effect sizes, as the ability to replicate a study depends on the existence of similarly sized or larger cohorts with equivalent phenotypic measurements. Computational strategies such as cross-validation (e.g., splitting a cohort into training and validation sets) can be used, but at the cost of decreasing power to detect novel associations. Within-cohort replication cannot address biases present in that cohort (e.g., access to health care, prevalence of smoking), as every random subset of the cohort used in cross-validations suffers from the same bias.

Genetic association studies have historically focused on White populations; however, recognition that diverse backgrounds can improve discovery and fine mapping (19) has led to efforts to study more diverse populations and recruit participants of diverse genetic backgrounds in biobank cohorts (20, 21). In addition, rare variants are more likely to be population specific (19), meaning that diverse cohorts will also improve power for discovery of new targets. While most of the observed variation in effect size across populations can be explained by low power, allele frequency differences, or differences in LD structure (22), we cannot rule out the possibility that some variants may have effect sizes that vary between populations (23), pointing to a genetic effect that may be specific to a particular environment.

Even with biobank-scale primary analysis in diverse populations, replication in independent cohorts will remain an important tool to strengthen true gene-disease associations and weaken false ones (18). Representatives of the 60 human cohorts with the largest number of worldwide participants have formed the International 100K Cohorts Consortium (4), whose primary mission is to facilitate exchange of knowledge and best practices and to devise a strategy for sharing data.

However, even when replicated, genetic evidence alone is insufficient to provide a clear path to intervention. Such a path typically requires a more detailed understanding of the molecular pathways that lead to disease development and relies in large part on deeper phenotypic profiling of the relevant populations.

Deep phenotyping

Our ability to intervene on a putative target is greatly assisted by a molecular understanding of disease pathogenesis and progression. Gaining such an understanding is a difficult task, as it requires collection of longitudinal biochemical and physiological data in patients, cell cultures, and laboratory animals, along with intense analytical labor needed for interpretation. Recent technological advancements have facilitated data collection across various levels of human biology and have provided us with rich data sets describing the molecular, biomarker, and physiological states of numerous cells, tissues, and organs. The comprehensive collection of such multilayer phenotypic data is often referred to as “deep phenotyping.”

At the molecular level, global initiatives, such as ENCODE (24), NIH Roadmap Epigenomics (25), BLUEPRINT (26), and GTEx (27), have quantified DNA methylation, chromatin accessibility, and gene regulation across a myriad of tissues and cell types in the human body. Additionally, efforts have begun to profile tissues at the single-cell level in normal conditions (28), as well as various developmental (29) and environmental contexts (30).

At the biomarker level, it is now possible to apply metabolomic, lipidomic, and proteomic analyses to large cohorts and acquire a comprehensive catalog of molecular species found in biofluids (31–35) and the microbiome (36). These multi-omics techniques gather data in a nontargeted manner to capture a large fraction of the biochemical space, including known as well as unknown molecular entities. Unbiased longitudinal measurements of biomarkers, collected before and after disease manifestation, can be instrumental for identifying potential causal mechanisms of disease pathology and have a considerable impact on the results of clinical testing: a recent analysis suggests that the success rate of clinical trials can be doubled by inclusion of at least one biomarker in patient selection (37).

At the physiological level, radiological imaging techniques are starting to provide noninvasive, high-resolution anatomical and functional information across all aspects of human physiology. Efforts to link brain imaging data to genetic and outcome data have already yielded new biological insights (38, 39). In addition, passive data collection from wearable devices allows for physiological monitoring at high temporal resolution (40).

While these early phenotyping efforts have already produced an unprecedented wealth of information, it is safe to say that this is just the beginning. In the future, the phenotypic data collected for human populations have the potential to scale up along every possible dimension: depth (number of traits captured), width (number of time points sampled), and height (number of individuals profiled). Integrating these highly dimensional data sets with genetics and clinical outcomes will be key to refining our mechanistic understanding of disease and prioritizing actionable therapeutic hypotheses (Figure 2). However, such integration will be challenging given the current lack of coherent conceptual frameworks and appropriate modeling techniques.

Figure 2. Use of deep phenotyping to limit the number of intervention hypotheses.

Association between genetic variation, intermediate traits, and outcomes. The large number of correlation connections (gray) can be reduced by introduction of sparsity into a network structure via Bayesian network inference (blue). Spurious correlations can be removed if outcomes are explained better by a different path through the network. Mendelian randomization (red) can also identify causal connections by using genetic variation within populations. Interventions on a red node or a blue node are more likely to succeed, as they mediate a path to a disease.

We are still in the early phase of our efforts to collect deep-omic phenotypes at scale, but the issue of replication should be carefully considered here as well. Apart from a few clinical biomarkers that have been routinely measured across large cohorts with carefully validated standardized procedures, no such standardization exists for most nonclinical biomarker measurements. Robust deep-omics measurement techniques that could be deployed on large cohorts are an active area of development (41, 42). Furthermore, application of these techniques across multiple cohorts would require a substantial multi-organizational effort to standardize sample preparation procedures, instrumentation, and quality control measures. These are important and necessary steps that will determine the usability of deep-omics data in the long run.

Integrative modeling and causal inference

The ultimate goal of computational modeling is to enable accurate predictions of a system’s behavior under perturbation. The complexity of biological systems, driven by a dense network of dynamic biochemical and regulatory interactions, has long hindered our ability to model them comprehensively. A variety of methods have been developed to make predictions based on correlations between molecular, physiological, and clinical measurements (43); however, transitioning from correlation to causation (a key ingredient for a successful clinical trial) remains a great challenge.

To address this challenge, a number of statistical approaches have been developed (44–46). These approaches, cumulatively referred to as mediation analyses, focus on the identification of intermediate phenotypes (e.g., biomarker levels in plasma) that might explain the association between an exposure (e.g., drug treatment) and an outcome (e.g., disease). The identification of such phenotypes is often critical for uncovering molecular mechanisms and provides considerable assistance in drug development. Among the various methods for mediation analysis, Mendelian randomization (MR) is of particular interest, as it takes advantage of natural genetic variation in human populations (46), allowing for stratification of individuals in a way that is analogous to a random assignment in clinical trials (47). In the most basic MR design, a robust genetic association of a variant (e.g., in the PCSK9 locus) with an intermediate phenotype (levels of LDL cholesterol) can be used as a proxy to estimate the effect of a drug exposure (statins) on an outcome (cardiovascular disease) (48–50).

Provided that the underlying assumptions are met, MR methods offer a powerful tool for identifying potential causal relationships. For instance, the directionality of the relationship between levels of LDL cholesterol and risk of coronary artery disease (CAD), as predicted by MR (51), is consistent with the results of clinical trials (52). Similar results have been obtained for HDL cholesterol and CAD (53–55), as well as vitamin D and type 2 diabetes (56–58). Causal predictions are particularly useful when clinical testing would be impractical or unethical — for instance, the effect of alcohol consumption on cardiovascular traits (59, 60). Encouraged by these early successes, the development of MR methods is an active area of research. One particular challenge is that many genetic associations have small effects on intermediate phenotypes, which can lead to inaccuracies in the causal effect estimates (61).

Given the heterogeneity of information and types of regulation within biological networks, multiscale models will be required to integrate information from different levels of biology (62, 63). A complete molecular description of network structure underlying human physiology does not exist, and we are left with all-by-all correlation structure between genes, proteins, metabolites, and physiological measures constructed from big data. A variety of methods (e.g., Bayesian networks, partial correlation networks) have been developed to reduce the complexity of these networks by removing spurious connections that are explained best by other connections in the network (43). These techniques produce sparse representations of a network where edges are the most likely causal relationships (Figure 2). We have not seen wide adoption of these methods to target discovery, but, combined with genetics, they could be of high value to clinical research.

Collectively, causal inference and integrative modeling can be instrumental in reducing the number of possible gene-disease associations to a more actionable subset. However, despite their early successes, current prediction methodologies are still in their infancy, and the predictions made by such methods can only be firmly established using experimentation and clinical testing.

Experimental medicine

Considering the inability of current computational models to predict accurately the effects of therapeutic interventions, our primary path to knowledge is through experimental testing. Animal models of disease are often used to perform rescue experiments that test the ability of a candidate intervention to revert or at least ameliorate the disease phenotype (64, 65). While proven to be extremely useful, animal models have known limitations due to their inability sometimes to recapitulate the physiological changes and response to therapy observed in humans (66, 67).

An alternative strategy is to learn about human physiology from individuals that carry loss-of-function mutations in promising target genes and can therefore be thought of as models of inhibition of those targets (68). Such “natural experiments” are found in populations that underwent strong founder events or elevated rates of consanguineous marriages that resulted in high rates of homozygosity for rare mutations, including those predicted to have severe loss-of-function effects (69–71). Deep phenotypic profiling of these individuals, who are effectively knockout models for one or more genes, can be used to investigate the physiological effects and safety implications of gene product inhibition, gain greater insights into biological pathways, explore gene modifiers, and establish gene dosage effects on disease outcomes. Early analyses of naturally occurring human knockouts in European and Pakistani populations have validated known drug targets and suggested new routes for intervention (e.g., NAV1.7 and pain, CCR5 and HIV, APOC3 and HDL cholesterol) (69, 72, 73).

Although affording important insights into human biology (74), the genomics of large-scale data, including those derived from human knockouts, is only one hand clapping. Many pathologies and indeed drug responses arise from genetics, environment, and time-dependent interactions. The full extent of these nongenetic contributions is hard to approximate, but most estimates suggest that somewhere between 60% and 80% of phenotypic variation is environmental (75). In one example, a maximal estimate of the contribution of genomics to variability in drug response in young healthy volunteers was approximately 30% (76). Data recorded on drug administration in the electronic health record are rarely confirmed by measurements of drug exposure or other objective assessments of adherence.

Understanding both the interindividual differences in network perturbations consequent to target engagement and how variable environmental conditions (77), including time of dosing (78), alter drug response within an individual is intrinsic to the development of a more precise approach to medicine. Such insights are dependent on experiments that test interventions in small groups of human subjects under basal and perturbed conditions in controlled environments. These experiments can afford deep and unbiased phenotypic characterization of their molecular, biomarker, and physiological responses. The data can provide unique insights into drug efficacy (79), identify biomarkers of drug susceptibility and response (80), and help refine patient selection for inclusion in clinical trials (81).

Importantly, experimental medicine addresses the greatest vulnerability in drug development — an accelerated passage through phase II, leading to poor estimates of drug efficacy due to shallow response measurements and low power, and thus to poor decisions about proceeding to phase III, the longest, most expensive, and most labor-intensive stage of clinical trials (82, 83). This bidirectional integration of such deep phenotypic data from experimental medicine with large observational data sets — i.e., human phenomic science (84) — promises to improve our understanding of drug action and variability in drug response. This knowledge will refine patient selection for large and expensive phase III trials, potentially limiting the size, duration, and cost of drug development.

Conclusion

The highly regulated world of clinical trials relies on blinded randomized experiments to test whether a single intervention is a safe and effective means to improve human health. Recent advances in genomic technologies, biochemistry, imaging, and automation are generating an unprecedented amount of data that, in turn, produce an overwhelming number of therapeutic hypotheses that could be taken into clinical trials. Importantly, while mining big data does create a deluge of hypotheses, it also offers a path to navigate through them. Large data sets, along with rigorous computational methods, enable validation, integration, and causal analysis of multiple lines of evidence to support or refute a hypothesis, improve our understanding of disease mechanisms, and identify a development path most likely to succeed. Furthermore, small-scale validation experiments afforded by experimental medicine provide a better understanding of candidate interventions and help to design better strategies for large-scale clinical testing. Outstanding challenges include the development of capacity for replication of experimental medicine data sets and the recognition of and adjustment for sources of bias in cohort data (e.g., ethnic and social diversity). Ultimately, the incorporation of big data and experimental medicine approaches into a standard practice should help reduce the failure rate of clinical trials and lower the cost of drug development.

Version 1. 01/13/2020

Electronic publication

Version 2. 02/03/2020

Print issue publication