Are we ready for causal discovery in biological systems using deep learning?

Abstract

The field of causal discovery has advanced considerably over the past three decades, in terms of perspectives, computational methods, and foundational concepts. Nevertheless, their application to biological systems that are commonly found in nature (i.e. large-scale, self-regulating), continues to face significant challenges. In this regard, we highlight emerging approaches that go beyond the traditional assumption of global acyclicity, instead leveraging efficient and scalable neural methods to infer pairwise causal relationships, directly from the data. Nonetheless, there remains five key technological hurdles, which must be overcome, to realize the deeper understanding and stronger inference biological causal networks promise.

Advancements in causal discovery for biological systems

Causality remains a foundational area of inquiry across diverse scientific domains [1, 2]. Nonetheless, a series of advances in the last three decades, as to elemental concepts, perspective, and computation, have increasingly permitted large-scale causal discovery in biological systems. Emerging research is now pivoting toward powerful, efficient, and scalable neural approaches that can map individual causal edges between variables from input data, in lieu of the hard task of discerning entire candidate networks. Such capability, central to higher-order intelligence (Supplementary Perspectives 1 and 2), however remains an open question at the heart of biology, one best interpreted through the lens of developmental trends thus far. To this end, the techniques proposed to date may be categorized as follows, while acknowledging the field’s inherent complexity and disorderly evolution:

1. Those fundamentally requiring Bayesian statistical formalism for causal inference

   • These methods [3–24] represent diverse, yet data and compute-hungry methods for searching plausible networks in entirety, and as such, face the intractable issues of a super-exponentially growing search space () (as the number of variables n increases), coupled with the challenge of score-equivalent but graphically distinct solutions.

   • Such classical techniques are subsequently superseded by innovatively embedding the Bayesian acyclicity constraint into the continuous objective search function [25], thereby enabling efficient exploration by powerful optimization algorithms. By notably allowing for stochastic gradient descent algorithms, the new formulation further unlocks neural approaches with novel capabilities [26–40]. Put together, the development enhances potential scalability to larger systems.

   • Many of these improvised methods also circumvent the fundamental limitation of using observations to infer causality, by leveraging on intervention data (Supplementary Perspective 2). Although some even evolved to use supervised learning to achieve more efficacious discovery (e.g. [33, 34]), the vast majority hold onto the assumption of global directed acyclicity (absence of feedback loops), an Achilles’ heel for causal discovery in self-regulating biological systems. Only Efficient Neural Causal DiscOvery (ENCO) [29] could discard the assumption under certain mild conditions.

2. Others rephrasing causal discovery as a simpler problem on binary classification

   • Given the criticality of molecular feedback loops for cellular functions, such as timing and decision-making processes (Supplementary Perspective 2), some later lineages escape the clutch of global directed acyclicity by abandoning the Bayesian notion at its core. Rather, they focus on using supervised learning to discriminate the presence or absence of directed edge for all pairs of variables [2, 41–44]. These techniques as such need not explore a super-exponentially growing search space (as systems get larger) and yet can provide more reliable inference due to supervision.

   • To exemplify this development, a recently-developed method, deep discriminative causal learning (D²CL) [2], has been reported to be scalable up to 50 000 variables for artificial systems, comparing favorably to that of classical state-of-the-art methods, e.g. 48 variables for Structural Discovery from Interventions (SDI) [30], 100 for Differentiable Causal Discovery with Interventions (DCDI) [27], and 1000 for Differentiable Causal Discovery of Factor Graphs (DCD-FG) [28]. Furthermore, D²CL requires as little as 10³ samples (both observation and interventional) to do so, in contrast to DCD-FG requiring at least 5 × 10⁴ such samples. Along the same line, DCDI needs at least a magnitude more samples (10⁶) and interventional experiments on all nodes; ENCO also has sample size requirements and system scalability within the range of its peers [2].

   • Inference-wise, D²CL surpasses ENCO, DCD-FG, and Scalable Causal Learning (SCL) [42], over a broad range of signal-to-noise ratio, as to both direct and ancestral causal effects in a nonlinear 1500-variables system. D²CL likewise outperforms several first-generation classical methods.

   • To be clear, D²CL bests other techniques in the regime that it has been designed for (i.e. where biomedical applications lay):

     • – one in which data dimensionality far exceeds sample size

     • – but with existing knowledge of a subset of causal relationships

     D²CL aptly leverages the latter for supervised learning to mitigate the curse of the former, using powerful and highly scalable neural approaches. As a trade-off, D²CL does not seek to infer the specific data-generating model, unlike Bayesian methods that require many more samples than data dimensions to do so. Put differently, the latter are designed for a different use case: where system dimensions are markedly less than the number of available samples, and little is known about its cause and effect.

   • D²CL however still uses assumptions that may be inconsistent with the behavior of biological systems. It assumes that each causal factor operates independently, and that by varying it, a response may be observed in its targets. Furthermore, the factors are presumed to act similarly to enable the learning of unknown causal edges from the knowledge of others. While it remains unclear as to the prediction cost of these assumptions, the authors have demonstrated good performance across both yeast gene deletion datasets (ROCAUC 0.79–0.85 across scenarios) and human CRISPR-based intervention datasets (ROCAUC 0.65–0.73). Among the compared methods, only SCL (of the same category as D²CL) achieves similar results on the yeast dataset (ROCAUC 0.75–0.77).

   • While D²CL focuses on inferring causal relationships in a system by using data and causal knowledge collected on, another emergent algorithm, Causal Structure Induction via Attention (CSIvA) [44], leverages transformer to supervise learning from diverse synthetic systems to unseen or even natural ones (i.e. meta-learning). Like D²CL, CSIvA has been shown to outperform DCDI and ENCO in inference (e.g. for up to 80 variables, different graph densities, and linear and nonlinear datasets) [44].

   • Still, the algorithm relies on intervention data and assumes the full network is identifiable upon sufficiently rich interventions on each causal factor. This again implies that the causal factors are to act independently to some extent, so that their manipulation generates an observable change in their targets. Yet, even when the synthetic data in case studies are designed as such, CSIvA requires tremendous amount of data: for a system of only 80 variables, CSIvA requires a diversity of 40 000 networks and 1500 samples for each of them. In addition, the topology of the network-to-be-inferred must be sufficiently characterized to generate training data with matching distribution. On this, the authors reported instances of difficulties in generalizing learning across different types of networks. Larger and denser graphs, alongside some data-generating functions, are also found to be harder to learn from.

   • Overall, CSIvA still represents a novel approach by harnessing attention mechanism [45] to pick up variations among samples and among factors that inform specific causation. In this regard, there is potential to pick up more causal information, such as among timepoints and among data modality with valid mechanisms (Fig. 1b).

   • Compute-wise, although CSIvA and D²CL may take longer to train than ENCO takes to infer for small systems (<1000 variables), the time spent can be amortized over multiple usages while inference only takes minutes [44]. (DCDI takes many more hours to compute).

Figure 1.

Advances necessary to enable meaningful, widespread applications of causal discovery tools for biological systems. (a) Key technological hurdles that must be overcome to realize the deeper understanding and stronger inference biological causal networks promise. These include two with significant recent progress (i.e. regarding feedback loops and scalability) and another five going forward. (b) Variations that may inform causal inference: Among factors within single modality (Type I), among factors of different data modality with regulatory relationship consistent with the Central Dogma (Type II), individual factors changes across samples (Type III), and individual factors changes across timepoints (Type IV). An example of each type is depicted in the diagram. Currently, only Type I and III have been implemented [44]. Here is the grand challenge: To develop a general architecture that can flexibly customize to the number and nature of interaction among data modalities.

Moving forward

The development of CSIvA and D²CL signifies that technological breakthroughs may be on the horizon, as deep learning classifiers are expected to enable causal discovery in many more biological systems, that are commonly found in nature (i.e. those that are of a much larger scale and self-regulating), than is possible previously. However, there is currently a dearth of rigorous work on the theoretical limitations of emerging deep learning techniques, the boundary conditions of their application, or potential failure modes, as computer scientists are only beginning to explore their applicability. Furthermore, there are at least five technological hurdles that must be overcome before humanity can realize the deeper understanding and stronger inference (Supplementary Perspectives 1–6) that causality promises (Fig. 1a):

• There is a need to distinguish between inhibiting and activating interactions to support adjacent analysis (e.g. control theory, dynamic modeling), like in many other scientific domains. This will allow for the derivation of sharper and more biological insights, as articulated in Supplementary Perspective 2. Notably, such a large-scale approach is non-existent, perhaps due to the additional complexity of the problem.

• In addition, a focus on identifying direct causation via elemental interactions (e.g. physico-chemical [46]) would prepare the ground for inferring the mechanisms involved (e.g. reactions) and their governing equations. See Supplementary on the importance of using first-principles for developing a true cellular digital twin [47] (Supplementary Perspective 2), and an example of how mechanism informs the right corrective action (Supplementary Perspective 5). In this regard, an ambitious effort is ongoing in DeepMind to predict the ‘interactions of all of life’s molecules’. However, it is unclear what kind of approach or data would be universally applicable across the different classes of biomolecules, and if such data is already available or could be generated in sufficient quantities, given the high contextuality and dynamicity of the interactions.

• For cellular systems, the grand challenge is to use multi-omics data to infer the physico-chemical regulatory mechanisms, in line with the Central Dogma and the order of events (Fig. 1b). As computer scientists are still exploring the theoretical and practical limits of causal inference using single-modality data, very limited effort has been made to extend these methods to multi-omics data. Such an approach will however allow for the construction of a virtual cell, whose implications are elegantly described by Demis Hassabis in one of his LinkedIn post:

  ‘Imagine running “in silico” experiments orders of magnitude faster than in a wet lab. Scientists could rapidly test hypotheses, model complex pathways and see how a drug affects a cell. It would be an incredible boon not only for fundamental biology but also for medicine.’

  Undoubtedly, the challenge is enormous. However, if we are to turn the problem on its head, embedding known regulatory structures as first-principle constraints [48] may effectively ease issues of sample size requirement, model identifiability, noise robustness, and overfitting. The core problem may however be perennial, as larger, more complex, and heterogeneous biological networks will still demand even more and better data (and compute), putting a strain on infrastructure, and demanding further innovations such as in quantum computing to arrive at the solution efficiently. (We note somewhat related methods have been proposed to explore generic mediators of causation, which can use but however does not distinguish between the different types of omics data, e.g. [49, 50]).

• Synthetic data should strive for contextual realism by closely emulating fundamental laws and real noises (e.g. heteroscedastic noise for biological data) to foster confidence [51]. Also, as methods developed to date are based on data generated from artificial models as the ground truth, little is known about their potential applicability and generalizability to real-world problems. Compounding the issue, many natural laws are still poorly understood, creating a potential chicken-and-egg problem.

• Current sample requirements for efficacious causal prediction (>> 1000 [2, 44]) far exceed what most biological studies can provide [48], except in the context of some single-cell omics. Yet even these are largely observational rather than interventional, aside from a few niche cases [52]. Coupled with the field’s infancy, this means that no clinical applications or actionable biological hypotheses have yet emerged. To address the issue, advancement in the generation of omics datasets must be complemented with better rationalization and optimization of data requirement, such as by applying the Design-Build-Test-Learn cycle (Fig. 1a) and related recommendations given by Radivojević et al. [53].

We believe the ultimate solution (and hurdle) would have to effectively address several of these challenges at once.

Key Points.

• Causal discovery methods split between classical Bayesian approaches and scalable neural classifiers more suited for biology (e.g. causal structure induction via attention, deep discriminative causal learning).

• Five key hurdles for cellular systems: distinguishing activating and inhibiting interactions, inferring direct physico-chemical causation, multi-modal integration, synthetic data, and noise realism, as well as sample size optimization.

• Uniting systems biology expertise with machine learning will drive the next leap in the field.

Conflict of interest

None declared.

Funding

This work was supported by the core research budget of Bioinformatics Institute, Agency for Science, Technology and Research (A^*STAR).

Data availability

There is no new data associated with this article.