Tree-based additive noise directed acyclic graphical models for nonlinear causal discovery with interactions

ABSTRACT

Directed acyclic graphical models with additive noises are essential in nonlinear causal discovery and have numerous applications in various domains, such as social science and systems biology. Most such models further assume that structural causal functions are additive to ensure causal identifiability and computational feasibility, which may be too restrictive in the presence of causal interactions. Some methods consider general nonlinear causal functions represented by, for example, Gaussian processes and neural networks, to accommodate interactions. However, they are either computationally intensive or lack interpretability. We propose a highly interpretable and computationally feasible approach using trees to incorporate interactions in nonlinear causal discovery, termed tree-based additive noise models. The nature of the tree construction leads to piecewise constant causal functions, making existing causal identifiability results of additive noise models with continuous and smooth causal functions inapplicable. Therefore, we provide new conditions under which the proposed model is identifiable. We develop a recursive algorithm for source node identification and a score-based ordering search algorithm. Through extensive simulations, we demonstrate the utility of the proposed model and algorithms benchmarking against existing additive noise models, especially when there are strong causal interactions. Our method is applied to infer a protein–protein interaction network for breast cancer, where proteins may form protein complexes to perform their functions.

1. INTRODUCTION

Causal discovery is a critical task in statistics and machine learning. It aims to infer causal relationships among random variables based on data. This task has numerous applications in various domains, including systems biology and social science. Directed acyclic graphs (DAG) are commonly used for representing these causal relationships, where arrows between nodes indicate direct causal relationships. Two assumptions are frequently made in graph-based causal discovery (Pearl, 2009): the causal Markov assumption, positing that the joint distribution of random variables respects conditional independencies encoded in the causal graph, and causal faithfulness, which is the converse of the causal Markov assumption. Based on these assumptions, the constraint-based PC (named after its creators Peter Spirtes and Clark Glymour) algorithm (Spirtes et al., 2000) conducts conditional independence tests to infer causal relationships. However, this type of method presents several limitations. First, it infers Markov equivalence classes rather than individual causal graphs, rendering some causal relationships inconclusive. Second, the faithfulness assumption can be violated with finite samples (Uhler et al., 2013), possibly leading to incorrect conclusions about causal relationships. Furthermore, conditional independence tests are generally not powerful statistical tests (Shah and Peters, 2020) despite some recent improvements (Tsamardinos et al., 2006; Colombo and Maathuis, 2014) on learning procedures.

There have been substantial explorations of structural causal models for causal discovery, particularly for continuous random variables. They encompass various causal models with distinct choices of structural causal functions and noise distributions. Importantly, several studies have shown that structural causal models are causally identifiable under additional distributional assumptions (Shimizu et al., 2006; Hoyer et al., 2008; Zhang and Hyvärinen, 2009; Peters and Bühlmann, 2014). Among them, additive noise models (ANMs, Hoyer et al., 2008) have attracted considerable attention. Peters et al. (2014) showed that these models are causally identifiable when structural causal functions are continuous (more precisely, their third-order derivatives exist), with certain peculiar exceptions such as linear models with unrestricted Gaussian noises. Additive models, Gaussian processes, or neural networks have been used to represent the structural causal functions. While additive models ignore interactions, Gaussian processes and neural networks are computationally expensive, require careful tuning, and are hard to interpret regarding the inferred causal mechanisms (eg, which set of variables has causal interactions).

Tree-based models have demonstrated remarkable success in various supervised machine learning tasks, such as classification and regression (Breiman et al., 1984; Quinlan, 1993). Tree ensemble methods like boosted trees (Freund and Schapire, 1996), random forests (Breiman, 2001), and Bayesian additive regression trees (Chipman et al., 2010) have been proposed to improve the predictive performance beyond that of individual trees. These models have been adapted to address more complex problems, including survival analysis (Ishwaran et al., 2008), treatment effect estimation (Wager and Athey, 2018), and conditional density estimation (Elie-Dit-Cosaque and Maume-Deschamps, 2022; Li et al., 2022). However, despite these advancements, the potential of tree-based models for causal discovery remains unexplored. Our goal is to harness the power of tree-based models for causal discovery. This novel adaptation extends the traditional scope of tree-based models, enabling them to elucidate causal relationships among random variables under typical causal assumptions.

It is widely recognized that tree-based models and their ensembles can accurately approximate complex functional relationships (Scornet et al., 2015; Ročková and van der Pas, 2020; Klusowski and Tian, 2023). This article shows that tree-based models can also provide valuable insights into causal mechanisms by quantifying the importance of individual variables as direct causes and measuring interactions among multiple variables. These viewpoints frequently arise in practical scenarios such as protein networks, where proteins are known to form protein complexes to perform functions. Tree-based models are intuitive and suitable for modeling the underlying structural causal function within the framework of ANMs, as trees are easy to compute and do not require extensive tuning. However, since existing identifiability theories of ANMs require structural causal functions to be continuous and smooth, they cannot be directly applied to tree-based ANMs, which have piecewise constant structural causal functions. Therefore, in this paper, we show that the proposed tree-based ANM is identifiable under mild regularity conditions.

We summarize our contributions as follows. First, we present a novel extension of causal identifiability theories for ANMs, incorporating piecewise constant (non-smooth) structural causal functions. It justifies the use of tree-based models in causal discovery. Second, the proposed tree-based ANMs can capture causal interactions. To demonstrate its impact on causal discovery, we consider a simple illustrative example of 3 variables such that with (eg, proteins and form a protein complex to influence protein ). Details of the simulation can be found in Web Appendix A. As shown in the left panel of Figure 1, the proposed tree-based model correctly identified the true causal graph as the sample size increased, measured by the structural intervention distance (SID, Peters and Bühlmann, 2015), whereas the causal additive model (CAM) (Bühlmann et al., 2014) failed as there was no main effect to capture (ie, the presence of either protein or protein alone does not affect protein ). Third, while ANMs estimated with Gaussian processes or neural networks can also account for interactions, they are not nearly as interpretable as trees, which can pinpoint exactly which set of variables has interactions; see, for example, Figure 5 in our application. Fourth, trees are also computationally more feasible than Gaussian processes and neural networks. The former is inherently slow because of the covariance kernel calculation when the sample size is large, whereas the latter often requires extensive tuning and a large sample size to perform well. The advantages of the proposed tree-based ANMs compared to other existing ANMs are summarized in the right panel of Figure 1.

FIGURE 1.

Left panel: A simple illustrative example showing the importance of incorporating interactions. The average SID with the standard error is plotted against the sample size. Right panel: Comparisons of tree-based models, additive models, Gaussian processes, and neural networks.

FIGURE 5.

The estimated breast cancer protein–protein interaction network. Colored ellipses indicate causal interactions among the encircled proteins on their targets, which are indicated by the same color.

The rest of the paper is organized as follows. We introduce our tree-based causal model and provide causal identifiability results in Section 2. Section 3 presents 2 algorithms for causal structure learning with the proposed tree-based ANMs. We conduct extensive simulations and a real-data application in Sections 4 and 5, respectively. We conclude by discussing future directions in Section 6.

2. MODEL

Let be a d-dimensional random vector. For any , let be the sub-vector of X indexed by A. Denote the domain of as . Throughout the paper, we assume there are no unmeasured confounders.

2.1. Structural causal models

Let be a DAG, which consists of a set of nodes representing the elements of X and a set of arrows representing the direct cause-effect (parent–child) relationships among X. In particular, an arrow in means that is the direct cause or parent of (accordingly, is the direct effect or child of ). For each , denote the set of all its direct causes by . Cycles are not allowed in the sense that any variable cannot return to itself by following the arrows. Each DAG admits at least one topological ordering , which is a permutation of its node indices such that it must hold when . We use to denote a topological ordering such that . For example, for the rightmost DAG in Figure 2, both and are valid orderings, but is not because but . Throughout the paper, we assume nodes are labeled according to the topological ordering, that is, , and denote as the set of variables preceding .

FIGURE 2.

An example of 2 Markov equivalence classes of 3-node DAGs.

For any DAG and probability distribution , let and denote the sets of conditional independencies encoded by the graph (via the notion of d-separation) and the distribution, respectively. A Bayesian network is a probability model such that the joint distribution of X factorizes over DAG as a product of node-induced conditional distributions, that is, . The factorization is equivalent to the global Markov property, that is, . The distribution is faithful to DAG if there are no more conditional independencies implied by the distribution than are implied by the DAG: . Two DAGs and are Markov equivalent if they encode the same conditional independence relationships, that is, , and a Markov equivalence class contains DAGs that are Markov equivalent. For instance, the first 3 DAGs in Figure 2 are Markov equivalent encoding , while the last DAG encodes .

Often, for continuous random variables, a Bayesian network can be constructed through a structural causal model for , where are independent noises or exogenous variables. Each structural causal function is deterministic, and it only depends on the set of direct causes and the noise . ANMs restrict the form of structural causal functions by assuming the noises to be additive:

where with a slight abuse of notation, we overload the definition of to emphasize that depends on via only. To avoid ambiguity or degeneracy in definition, we require that is not constant in any of its arguments. Assuming that and then, the node-induced conditional distribution is given by . Bühlmann et al. (2014) proposed the CAM for which causal functions are further restricted to be .

2.2. Tree-based ANMs

In this paper, we propose to use trees to model structural causal functions. Given the causal graph or the topological ordering, learning the causal model is equivalent to constructing d regression models, where each variable is regressed on the corresponding direct causes or preceding variables. Regression tree partitions the domain of each into distinct regions , giving rise to piecewise constant causal functions:

where are constants. Each depends on its preceding variables through its direct causes. As before, we overload as a function of the direct causes only and assume it is not constant in any of the direct causes, that is, for any , there exist and , which only differ in its jth element, such that . The tree construction proceeds recursively, guided by certain criteria such as minimizing the mean squared error, to find the partition and associated partition-specific constants. At the end of the tree construction process, pruning is used to sequentially collapse nodes in the tree for which the optimal depth is determined by the cost complexity pruning (also known as the weakest link pruning) commonly used in decision trees (Hastie et al., 2009), to prevent overfitting and improve interpretability.

However, in practice, the causal graph or the topological ordering is often unknown and needs to be inferred from data. This poses a more challenging unsupervised learning task as it requires the determination of causal directions without any prior knowledge. Furthermore, learning the causal structure becomes even harder since there can exist multiple orderings, or DAGs, that induce the same observational distribution with distinct causal interpretations (Spirtes and Zhang, 2016). In other words, a causal model may not be identifiable. However, we show that the proposed tree-based ANM is generally identifiable. Note that existing causal identifiability theories of ANMs (Hoyer et al., 2008; Peters et al., 2014) assume structural causal functions to be continuous and smooth (eg, their third-order derivatives exist). Hence, they cannot be directly applied to the proposed tree-based ANMs with discontinuous structural causal functions. Consequently, our identifiability result imposes distinct conditions on the causal function and noise distribution.

2.3. Causal identifiability

We now show that source nodes (ie, nodes without incoming arrows) can be recursively identified under mild conditions. That is, the causal ordering is identifiable.

Definition 1:

The causal ordering is identifiable if the inferred topological ordering is consistent with the true causal graph.

Proposition 1:

The causal ordering is identifiable if the additive noises follow independent Gaussian distributions.

Proofs are provided in Web Appendix B. The Gaussianity assumption is adopted by many existing methods (Bühlmann et al., 2014; Peters and Bühlmann, 2014; Li et al., 2023), but we show this assumption can be relaxed to a less restrictive version later. This condition is introduced to avoid degenerate cases, where the distribution of the cause and effect becomes symmetric, and ensures that when some direct causes are not conditioned on, the residual does not follow the assumed Gaussian distribution but rather follows a mixture thereof because of the tree partition. This allows recursive identification of the source node.

Proposition 2:

The causal ordering is identifiable if the following condition holds:

Condition. For the distribution of any noise , when , , and , there exists such that

This condition is satisfied by many exponential families and nonparametric distributions (Lindsay, 1995; Aragam et al., 2020) because of the identifiability of mixture models. In particular, Gaussian noise is a scenario where the condition is fulfilled. This is a non-degeneracy condition requiring that any mixture of the noise distribution not revert to the original noise distribution. We demonstrate these propositions with a trivariate example.

Example 1:

Consider the trivariate model with the graph given in Figure 3. Assume that for . Then marginally, we have , and

Since is the direct cause of and and is the direct cause of , there must exist such that for and . Therefore, we deem the source node since it follows a Gaussian distribution, whereas and follow mixtures of Gaussians. Next, conditional on , we have

Since is the direct cause of in addition to , there must exists such that and . Therefore, we deem the source node after removing , because follows a Gaussian distribution whereas follows a mixture of Gaussians conditional on .

FIGURE 3.

A trivariate example demonstrating source node identification regarding residual distributions.

As expected, our identifiability condition is quite distinct from those in Hoyer et al. (2008) and Peters et al. (2014), as we consider a distinct class of structural causal functions. We demonstrate this distinction with the following example:

Example 2:

In this example, we demonstrate the distinction between continuous (existing methods) and piecewise constant (our method) structural causal functions in terms of identifiability for the bivariate case. Consider 2 random variables and . In this case, the key point of causal identification is whether the combination of the causal function and noise distribution creates an asymmetric relationship between the cause and effect that distinguishes the 2 models and .

In the case of continuous structural causal functions, linear causal functions with Gaussian noises are well-known to be non-identifiable because of their perfectly symmetric relationship (top-right panel of Figure 4). However, assuming non-Gaussian (Shimizu et al., 2006) such as uniform noises renders the causal relationship identifiable (bottom-right panel of Figure 4).

For piecewise constant structural causal functions, uniform noises play similar roles to Gaussian noises in the continuous case, leading to non-identifiable models. For example, consider and

where , , and H is an integer. In this case, both and follow marginally, and the relationship is perfectly symmetric (bottom-left panel of Figure 4). However, Gaussian noises render the causal ordering identifiable (top-left panel of Figure 4), because Gaussian noises in one direction give rise to mixtures of Gaussian noises in the opposite direction.

FIGURE 4.

Bivariate examples demonstrating relationships between causal functions and noise distributions that render the causal direction either identifiable (top left and bottom right) or non-identifiable (top right and bottom left).

3. STRUCTURE LEARNING ALGORITHMS

We introduce 2 structure learning algorithms, one based on recursive source node identification, and the other is a score-based ordering search. Here, we assume noises follow independent Gaussian distributions. We choose regression trees (Breiman et al., 1984) for construction, which can be extended to other trees such as classification trees for discrete data and tree ensembles for better causal predictions.

3.1. Recursive source node identification

We provide an algorithm that recursively finds source nodes (ie, the first node among undetermined nodes in the ordering), following the idea of our identifiability theorem. Intuitively speaking, when some directed causes are missing as predictors in the regression tree model, the regression residual follows a mixture distribution. For Gaussian noises, these residuals are non-Gaussian and can be quantified using normality tests (eg, Anderson–Darling test, Anderson and Darling, 1952).

To be concrete, denote the current ordering at step S by , which is an ordered set, and denote the set of nodes to be ordered by . We set and to begin with. For any node to be ordered, we construct a regression tree that regresses node on nodes that have been ordered and let denote the corresponding regression residual. Then, we select the node with the maximal Gaussianity:

where is a measure of Gaussianity, and set and . The algorithm terminates in steps. We choose to be the P-value from the Anderson–Darling test of Gaussianity. Although the true causal DAG may have multiple orderings, the proposed algorithm is guaranteed to converge to one of them given perfect test results.

3.2. Greedy score-based search

We notice that the recursive procedure tends to accumulate and propagate errors with a finite sample, as also previously mentioned by Peters et al. (2014). To address this limitation, we turn to a score-based search procedure. Furthermore, we construct our score function and search over the space of orderings, which is known to have better empirical performance compared to searching over the space of DAGs (Teyssier and Koller, 2012). We discuss the choice of the score function and search procedure in the following paragraphs.

We score any given ordering by the Bayesian information criterion

where n is the sample size, is the residual sum of squares for the th regression tree constructed according to the ordering , and represents the number of leaf nodes in the th tree. In particular, for each , the th tree is constructed by regressing on its preceding variables and pruning out unnecessary splits with cross-validation. The score function measures the soundness of a candidate ordering using regression residuals and penalizes the model complexity by the total number of splits, which accounts for both graph complexity and tree complexity. Intuitively speaking, when we fit all orderings with respective regression models, the structural causal functions under the true ordering can often be approximated by relatively simple trees, whereas the structural causal functions under any wrong ordering can be more complex for the equal goodness of fit.

However, a complete enumeration of all possible orderings becomes impractical beyond a few nodes due to the factorial growth of the ordering space, necessitating a greedy search procedure. The greedy search algorithm depends essentially on a set of operators that iteratively refine the current ordering, and the choice of these operators can significantly influence the performance of the algorithm. We consider a recent algorithm proposed by Solus et al. (2021). The algorithm depends on 2 user-chosen parameters: the number of runs (initializations) and the search depth. Increasing these 2 parameters can improve its performance at the expense of additional computational resources. The algorithm proceeds as follows:

1. Initialize an ordering and denote its associated DAG as , which is obtained by running a regression tree algorithm (fitting and pruning) for each node given its preceding variables in the ordering.

2. Use a depth-first search procedure starting from the initial ordering to find an ordering with that is connected to through a sequence of orderings within the search depth. Each ordering is obtained from by performing a covered arrow reversal on the graph to obtain and then selecting an ordering consistent with DAG . A covered arrow reversal produces a DAG from by reversing a covered arrow (an arrow is covered if in ).

3. If no such exists, terminate the algorithm and output the final ordering . Otherwise, set and repeat the second step.

4. Repeat the entire procedure with multiple runs and select the ordering with the minimum score as the final output.

3.3. Parameter choices and computational considerations

We focus on the greedy score-based search algorithm, which had better empirical performance than the recursive source node identification procedure. The greedy algorithm involves 2 components: the tree construction and the greedy search procedure. We use the R package tree with pruning steps and 10-fold cross-validation to implement regression trees. Due to the greedy nature of the search algorithm, increasing the number of runs and search depth improves the chances of escaping local optima, but at the cost of higher computational expense. We found that setting the number of runs to 20 and the search depth to 5 provided satisfactory performance in simulations (additional comparisons are provided in Web Appendix C.3). This choice is similar to that of Solus et al. (2021).

Besides better interpretability, the proposed tree-based causal discovery method is, ceteris paribus, computationally more efficient than those based on Gaussian processes and neural networks simply because of the computational advantages of trees over Gaussian processes and neural networks in function estimation. For example, for a regression problem with n samples and d covariates, regression trees have a complexity of , Gaussian processes have a complexity of , and neural networks have a complexity of , where E is the number of training epochs, L is the number of layers, and represents the number of neurons in the th layer. In addition, neural networks may be hard to tune. Some empirical comparisons are provided in Web Appendix C.4.

4. SIMULATIONS

To evaluate the performance of the proposed method, we conducted extensive simulations. The main simulation presented here serves to highlight the importance of modeling interactions in capturing the underlying causal relationships accurately. Web Appendices C.1 and C.2 present many additional simulation scenarios, including those that focus on bivariate causal discovery on various benchmark datasets (eg, nonparametric noises) and those with piecewise constant functions as true structural causal functions. Since the greedy search procedure performed consistently better than the recursive procedure, we only report the performance of the greedy search procedure in the paper and term it tree-GSBIC, short for tree-based causal ordering Greedy Search using the Bayesian Information Criterion. The recursive procedure is demonstrated in the Web Appendix.

4.1. Data generation

We varied the number of variables and considered the Erdős–Rényi graph with d (sparse) and (dense) expected numbers of edges. We assumed each true structural causal function to be a multivariate polynomial (not piecewise constant)

where if and otherwise. We scaled each variable to the range before applying the polynomial transformation. We sampled each polynomial coefficient from and with equal probability, and randomly set the coefficients of main effects (ie, the constant before for ) to 0 with probability so that some causal relationships may only have interaction effects. Lastly, we generated , where if and otherwise. We considered sample sizes and repeated the simulation 100 times for each scenario. For denser graphs, we additionally considered .

4.2. Competing methods

We compared the proposed tree-GSBIC with the CAM (Bühlmann et al., 2014) and the regression with subsequent independence test using Gaussian process regressions (GP-RESIT, Peters et al., 2014). As the Gaussian process regression function in their original implementation is no longer available, we replaced the function with the one in the R package kernlab.

4.3. Evaluation metrics

We report the SID (Peters and Bühlmann, 2015) and Matthews correlation coefficient (MCC) between the true and estimated DAGs. These 2 metrics serve to measure the accuracy of estimating individual DAGs. Lower SID and higher MCC indicate a more accurate estimation.

4.4. Results

The experimental result is summarized in Table 1. Overall, Table 1 shows that tree-GSBIC performed the best, followed by CAM. As the sample size increased, the advantage of tree-GSBIC became more substantial. This is not surprising because a large sample size allows trees to capture complex interaction patterns, whereas CAM is limited by its additive assumption. GP-RESIT did not perform well, possibly due to the RESIT procedure. Therefore, we provide additional comparisons with the greedy DAG search procedure (Peters et al., 2014) using the independence-based score and Gaussian process regression (GP-GDS) because this approach only scales to .

TABLE 1.

Average SID and MCC based on 100 repeat simulations.

d	n	tree-GSBIC		CAM		GP-RESIT
		SID	MCC	SID	MCC	SID	MCC
Sparse Graph
20	200	26.62 (0.86)	0.628 (0.01)	38.92 (0.96)	0.626 (0.01)	66.17 (0.98)	0.292 (0.01)
20	500	14.53 (0.59)	0.736 (0.00)	38.80 (0.64)	0.673 (0.00)	47.02 (0.43)	0.505 (0.01)
20	1000	10.60 (0.34)	0.842 (0.00)	34.85 (0.42)	0.698 (0.09)	44.78 (0.28)	0.578 (0.00)
30	200	96.22 (2.00)	0.612 (0.00)	146.6 (1.65)	0.596 (0.00)	239.3 (2.23)	0.137 (0.00)
30	500	45.43 (0.49)	0.722 (0.00)	143.7 (1.15)	0.659 (0.00)	209.5 (1.20)	0.244 (0.00)
30	1000	23.43 (0.33)	0.821 (0.00)	152.6 (0.89)	0.646 (0.00)	197.6 (0.80)	0.421 (0.00)
50	200	137.7 (2.28)	0.524 (0.01)	166.1 (1.56)	0.552 (0.01)	255.4 (1.78)	0.151 (0.00)
50	500	75.67 (1.23)	0.698 (0.00)	156.2 (1.38)	0.601 (0.00)	219.4 (1.04)	0.332 (0.00)
50	1000	47.45 (0.71)	0.809 (0.00)	142.8 (1.34)	0.648 (0.00)	204.9 (1.01)	0.343 (0.00)
Dense Graph
20	200	228.5 (2.42)	0.510 (0.01)	247.3 (1.80)	0.511 (0.01)	298.2 (2.20)	0.158 (0.01)
20	500	143.1 (1.47)	0.684 (0.00)	238.7 (1.16)	0.572 (0.00)	282.3 (1.25)	0.236 (0.00)
20	1000	100.3 (1.02)	0.805 (0.00)	206.7 (1.11)	0.621 (0.00)	257.5 (1.10)	0.342 (0.00)
20	2000	96.62 (0.61)	0.819 (0.00)	194.5 (0.72)	0.629 (0.00)	228.3 (0.67)	0.391 (0.00)
30	200	613.4 (2.67)	0.422 (0.00)	658.5 (3.58)	0.421 (0.01)	740.8 (2.89)	0.032 (0.00)
30	500	497.8 (2.20)	0.579 (0.00)	569.0 (3.35)	0.598 (0.00)	716.4 (1.48)	0.155 (0.00)
30	1000	362.1 (1.53)	0.699 (0.00)	561.8 (2.46)	0.588 (0.00)	672.8 (0.68)	0.261 (0.00)
30	2000	324.2 (1.49)	0.768 (0.00)	488.7 (1.87)	0.657 (0.00)	605.8 (0.61)	0.304 (0.00)
50	200	945.2 (6.71)	0.468 (0.07)	983.1 (6.41)	0.500 (0.00)	1000 (3.91)	0.024 (0.00)
50	500	856.7 (4.04)	0.647 (0.00)	959.3 (4.03)	0.660 (0.00)	1000 (2.32)	0.116 (0.00)
50	1000	816.1 (2.52)	0.748 (0.00)	911.3 (2.80)	0.716 (0.00)	1000 (1.71)	0.218 (0.00)
50	2000	601.8 (1.24)	0.779 (0.00)	850.8 (1.86)	0.721 (0.00)	1000 (0.94)	0.314 (0.00)

Standard errors are given in parentheses. Best performances are boldfaced.

In particular, we consider smaller-scale simulation scenarios where and . Data were generated in the same way as above using sparse graphs (ie, d edges in expectation). The result is reported in Table 2, which shows that tree-GSBIC outperformed the other methods with the smallest SID and largest MCC.

TABLE 2.

Average SID and MCC based on 100 repetitions for the small-scale simulation comparisons.

d	n	Tree-GSBIC		CAM		GP-RESIT		GP-GDS
		SID	MCC	SID	MCC	SID	MCC	SID	MCC
6	200	3.42 (2.21)	0.73 (0.08)	7.24 (3.46)	0.69 (0.11)	10.2 (4.21)	0.51 (0.23)	14.7 (3.54)	0.30 (0.25)
6	500	1.14 (2.02)	0.79 (0.16)	11.7 (3.45)	0.54 (0.15)	3.86 (3.44)	0.76 (0.19)	10.7 (6.24)	0.49 (0.32)
15	200	9.42 (3.25)	0.68 (0.08)	14.7 (4.16)	0.48 (0.09)	14.2 (5.40)	0.51 (0.10)	16.7 (4.39)	0.45 (0.13)
15	500	7.22 (2.49)	0.74 (0.07)	15.8 (3.61)	0.69 (0.15)	17.8 (4.73)	0.47 (0.15)	12.5 (3.43)	0.58 (0.09)

Standard errors are given in parentheses. Best performances are boldfaced.

5. APPLICATION: PROTEIN NETWORK

Breast cancer is the most frequently diagnosed cancer and the second leading cause of cancer-related deaths in women (Giaquinto et al., 2022). Despite substantial research endeavors over the past decades, its underlying mechanism has yet to be fully elucidated. In particular, unraveling and characterizing the complex protein–protein interaction network associated with breast cancer is an important but challenging task. Understanding protein–protein interactions is of utmost importance as proteins rarely function in isolation, and many molecular processes are related to the interplay between proteins. The advent of reverse-phase protein array analysis has provided a quantitative and cost-friendly means to examine protein abundance (Hennessy et al., 2010). We focused on reverse-phase protein array samples related to breast cancer. A panel of proteins known to be involved in 5 signaling pathways crucial to breast cancer, Estrogen, Notch, Wnt, MAPK, and PIK3-Akt, was selected for our analysis. The dataset that we used was sourced from the study conducted by The Cancer Genome Atlas Network (2012), and was assembled by Li et al. (2013).

To improve the robustness of the inferred network, we generated 50 bootstrap datasets and applied tree-GSBIC to each bootstrap dataset. The final protein network is reported in Figure 5, which includes arrows that appeared in at least 90% of the bootstrapped networks. Overall, our inferred network was consistent with known interactions and pathways in the literature. In particular, the inferred network highlights several key regulators, including proteins ESR1, PGR, IGFR1, EGFR, and KIT. These findings are consistent with existing knowledge, as these proteins are known to occupy upstream positions in the signaling pathway (please refer to the Kyoto Encyclopedia of Genes and Genomes [KEGG] reference breast cancer signaling pathway, which is summarized in Web Appendix D). Furthermore, the majority of arrow directions within the inferred network are consistent with the known directionality of the same KEGG reference signaling pathway, for example, ESR1 CCND1, EGFR NRAS, NRAS BRAF, MAP2K1 MAPK3, and EGFR/KIT PIK3CA. This consistency demonstrates that the inferred causal relationships among proteins align with the expected flow of molecular signals. Moreover, the co-regulatory role of proteins PGR/ESR1 from the Estrogen signaling pathway and IGFR1/EGFR from the MAPK signaling pathway suggests a potential cross-talk or synergy between these pathways, where these proteins act together and influence shared downstream targets. This is also consistent with previous research findings (Fagan and Yee, 2008; Skandalis et al., 2014). Also, the interactions between EGFR and IGFR1 within the MAPK signaling pathway, as well as between ESR1 and PGR within the Estrogen signaling pathway, are reported in the literature (Oliveira et al., 2009; Giulianelli et al., 2012). The identification of such co-regulatory relationships (ie, causal interactions) provides valuable insights into the interplay and coordination between distinct proteins or signaling pathways in breast cancer, highlighting the interconnected nature of these molecular processes. For completeness, we report the estimated networks by CAM and GP-RESIT in Web Appendix D.

6. DISCUSSION

Our work opens up several potential directions for future studies. Quantifying the uncertainty of causal interactions is an important problem. One solution is to use bootstrapping and count how frequently a particular interaction occurs among the bootstrapped samples, or calculate some other metrics such as the H-statistic of Friedman and the SHAP value (Friedman, 2001; Lundberg et al., 2020). Another solution is to replace our current tree model with a specialized Bayesian tree model such as Du and Linero (2019) that is designed for detecting interactions, which naturally provides uncertainty quantification.

Theoretically, we have only shown that if both the true and working structural causal functions are trees, the causal order is identifiable at the population level. If the true function is smooth and nonlinear, intuitively, our tree-based model may still be able to identify the true ordering as the sample size goes to infinity because the tree could approximate the smooth function arbitrarily well with an increasing depth. To rigorously prove it, however, may require a different proof technique because our current technique hinges on the finite mixture distributions induced by the finite tree partition. Capturing a smooth nonlinear function would require a tree with an infinite depth, and hence, accordingly, the proof needs to deal with infinite mixture distributions, which could be a future research topic.

In addition, we may consider speeding up the current search algorithms with preliminary neighborhood selections (Bühlmann et al., 2014). It would also be interesting to extend the proposed method to allow latent confounders and cycles. For the real data that we analyzed, breast cancer is a heterogeneous disease with various subtypes. One might learn subtype-specific network structures, as this could yield valuable insights into particular mechanisms of the disease subtypes.

Supplementary Material

ujaf089_Supplemental_Files

Web Appendices referenced in Sections 1–6 are available with this paper at the Biometrics website on Oxford Academic. The code that implements the proposed method is available at https://github.com/fangting-zhou/causal-tree and also online with this paper.

FUNDING

He’s research was supported by the National Key R&D Program of China (Grant No. 2023YFC3304701). Ni’s research was partially supported by NIH 1R01GM148974 and NSF DMS-2112943.

CONFLICT OF INTEREST

None declared.

DATA AVAILABILITY

The breast cancer dataset used in Section 5 of this paper is from The Cancer Genome Atlas Program (TCGA), at https://tcpaportal.org/tcpa. The breast cancer pathway information can be obtained from Kyoto Encyclopedia of Genes and Genomes (KEGG), at https://www.kegg.jp/kegg/pathway.html.