Leveraging graphical model techniques to study evolution on phylogenetic networks

Abstract

The evolution of molecular and phenotypic traits is commonly modelled using Markov processes along a phylogeny. This phylogeny can be a tree, or a network if it includes reticulations, representing events such as hybridization or admixture. Computing the likelihood of data observed at the leaves is costly as the size and complexity of the phylogeny grows. Efficient algorithms exist for trees, but cannot be applied to networks. We show that a vast array of models for trait evolution along phylogenetic networks can be reformulated as graphical models, for which efficient belief propagation algorithms exist. We provide a brief review of belief propagation on general graphical models, then focus on linear Gaussian models for continuous traits. We show how belief propagation techniques can be applied for exact or approximate (but more scalable) likelihood and gradient calculations, and prove novel results for efficient parameter inference of some models. We highlight the possible fruitful interactions between graphical models and phylogenetic methods. For example, approximate likelihood approaches have the potential to greatly reduce computational costs for phylogenies with reticulations.

This article is part of the theme issue ‘“A mathematical theory of evolution”: phylogenetic models dating back 100 years’.

1. Introduction

Stochastic processes are used to model the evolution of traits over time along a phylogeny, a graph representing the historical relationships between species, populations or individuals of interest, in which internal nodes represent divergence (e.g. speciation) or merging (e.g. introgression) events. In this work, we consider traits that may be multivariate, discrete and/or continuous, with a focus on continuous traits. Trait evolution models are used to infer evolutionary dynamics [1,2] and historical correlation between traits [3–5], predict unobserved traits at ancestral nodes or extant leaves [6,7], or estimate phylogenies from rich datasets [8–12].

Calculating the likelihood is no easy task because the traits at ancestral nodes are unobserved and need to be integrated out. This problem is very well studied for phylogenetic trees, with efficient solutions for both discrete and continuous traits [13–18]. Admixture graphs and phylogenetic networks with reticulations are now gaining traction due to growing empirical evidence for gene flow, hybridization and admixture [11,19,20]. Yet many methods for these networks could be improved with more efficient likelihood calculations.

The vast majority of phylogenetic models make a Markov assumption, in that the trait distribution at all nodes can be expressed by a set of local models. At the root, this model describes the prior distribution of the ancestral trait. For each node in the phylogeny, a local transition model describes the trait distribution at this node conditional on the trait(s) at its parent node(s). As each local model can be specified individually with its own set of parameters, the overall evolutionary model can be very flexible, including possible shifts in rates, constraints and modes of evolution across different clades [21,22]. Other models do not make a Markov assumption, such as threshold models [23,24], or models that combine a backwards-in-time coalescent process for gene trees and forward-in-time mutation process along gene trees [25]. We show here that some of these models can still be expressed as a product of local conditional distributions, over a graph that is more complex than the initial phylogeny.

These evolutionary models are special cases of graphical models, also known as Bayesian networks, which have been heavily studied [26]. The likelihood calculation task has received a lot of attention, including algorithms for efficient approximations when the network is too complex to calculate the likelihood exactly [26]. Another well-studied task is that of predicting the state of unobserved variables (ancestral states in phylogenetics) conditional on the observed data. We argue here that the field of phylogenetics could greatly benefit from applying and expanding knowledge from graphical models for the study and use of phylogenetic networks.

In §2, we review the challenge brought by phylogenetic models in which only tip data are observed and current techniques for efficient likelihood calculations. In §3, we focus on general Gaussian models for the evolution of a continuous trait, possibly multivariate to capture evolutionary correlations between traits. On reticulate phylogenies, these models need to describe the trait of admixed populations conditional on their parental populations. Turning to graphical models in §4, we describe their general formulation and show that many phylogenetic models can be expressed as special cases, from known examples to less obvious examples (using the coalescent process on species trees or species networks). We then provide a short review of belief propagation, a core technique to perform inference on graphical models, first in its general form and then specialized for continuous traits in linear Gaussian models. In §5, we describe loopy belief propagation, a technique to perform approximate inference in graphical models when exact inference does not scale. As far as we know, loopy belief propagation has never been used in phylogenetics. Section 6 describes leveraging belief propagation (BP) for parameter inference: fast calculations of the likelihood and its gradient can be used in any likelihood-based framework, frequentist or Bayesian. Finally, §7 discusses future challenges for the application and extension of graphical model techniques in phylogenetics. These techniques offer a range of avenues to expand the phylogeneticists’ toolbox for fitting evolutionary models on phylogenetic networks, from approximate inference methods that are more scalable, to algorithms for fast gradient computation for better parameter inference.

2. Complexity of the phylogenetic likelihood calculation

(a). The pruning algorithm

Felsenstein’s pruning algorithm [13,27] launched the era of model-based phylogenetic inference, now rich with complex models to account for a large array of biological processes including DNA and protein substitution models, variation in their substitution rates across genomic loci, lineages and time and evolutionary models for continuous traits and geographic distributions. The pruning algorithm gave the key to calculate the likelihood of these models along a phylogenetic tree, in a practically feasible way. The basis of this algorithm, which extends to tasks beyond likelihood calculation, was discovered in other areas and given other names, such as the sum–product algorithm, forward–backward algorithm, message passing and BP. In particular, the field of statistical human genetics saw the early development of such algorithms for models on pedigrees [28–30] including loops [31,32]. See section 9.8 in Koller & Friedman [26] for a review of the literature on variable elimination algorithms.

The pruning algorithm, which is a form of BP, computes the full likelihood of all the observed taxa by traversing the phylogenetic tree once, taking advantage of the Markov property: where the evolution of the trait of interest along a daughter lineage is independent of its past evolution, given knowledge of the parent’s state. The idea is to traverse the tree and calculate the likelihood of the descendant leaves of an ancestral species conditional on its state, from similar likelihoods calculated for each of its children. If the trait is discrete with four states, for example (as for DNA), then this entails keeping track of four likelihood values at each ancestral species. If the trait is continuous with a Gaussian distribution, e.g. from a Brownian motion (BM) or an Ornstein–Uhlenbeck (OU) process [33], then the likelihood at an ancestral species is a nice function of its state that can be concisely parametrized by quantities akin to the posterior mean and variance conditional on descendant leaves. Felsenstein’s independent contrasts (IC) [34] also capture these partial posterior quantities and can be viewed as a special implementation of BP for likelihood calculation.

BP is used ubiquitously for the analysis of discrete traits, such as for DNA substitution models (e.g. in RAxML [35], IQ-TREE [36], MrBayes [37]) or for discrete morphological traits in comparative methods (e.g. in phytools [38], BayesTraits [39], corHMM [40,41], RevBayes [42]). For discrete traits, there is simply no feasible alternative. On a tree with 20 taxa and 19 ancestral species, the naive calculation of the likelihood at a given DNA site would require the calculation and summation of $4^{19}$ or 274 billion likelihoods, one for each nucleotide assignment at the 19 ancestral species. This calculation would need to be repeated for each site in the alignment, then repeated all over again during the search for a well-fitting phylogenetic tree.

(b). Continuous traits on trees: the lazy way

For continuous traits under a Gaussian model (including the Brownian motion), BP is not used as ubiquitously because a multivariate Gaussian distribution can be nicely captured by its mean and covariance matrix: the multivariate Gaussian formula can serve as an alternative. For example, for one trait $Y$ with ancestral state $\mu$ at the root of the phylogeny, the phylogenetic covariance ${\displaystyle \mathbf{\Sigma }}$ between the taxa at the leaves can be obtained from the branch lengths in the tree. Under a BM, the covariance ${\displaystyle \mathrm{c}\mathrm{o}\mathrm{v}(Y_i,Y_j)}$ between taxa $i$ and $j$ is ${\mathrm{\Sigma }}_{ij}={\sigma }^2{t}_{ij}$ where $t_{ij}$ is the length between the root and their most recent common ancestor. The likelihood of the observed traits at the $n$ leaves can then be calculated using matrix and vector multiplication techniques as

$${\displaystyle (2\pi {)}^{-n/2}det|\mathbf{\Sigma }{|}^{-1/2}\exp \left(-\frac{1}{2}(Y-\mu {)}^{\mathrm{\top }}{\mathbf{\Sigma }}^{-1}(Y-\mu )\right).}$$ (2.1)

This alternative to BP has the disadvantage of requiring the inversion of the covariance matrix ${\displaystyle \mathbf{\Sigma }}$, a task whose computing time typically grows as $m^3$ for a matrix of size $m\times m$. It also has the disadvantage that ${\displaystyle \mathbf{\Sigma }}$ needs to be calculated and stored in memory in the first place. For multivariate observations of $p$ traits on each of $n$ taxa, the covariance matrix has size $m=pn$ so the typical calculation cost of (2.1) is then $O(p^3{n}^3)$, which can quickly become very large. For example, with only 30 taxa and 10 traits, ${\displaystyle \mathbf{\Sigma }}$ is a $300\times 300$ matrix. Studies with large $p$ and/or large $n$ are now frequent, especially from geometric morphometric data with $p$ over 100 typically (e.g. [43]) or with expression data on ${\displaystyle p>1000}$ genes easily, that also require more complex models to account for variation (e.g. within species, between organs, between batches [44,45]). Studies with a large number $n$ of taxa are now frequent (e.g. ${\displaystyle n>5000}$ in birds and mammals [46,47]) and virus phylogenies can be massive (e.g. ${\displaystyle n>500000}$ SARS-CoV-2 strains [48]). Viral continuous traits previously studied include virulence traits (e.g. ${\displaystyle n>1000}$ and $p=3$ traits in HIV [49,50]) and geographic data for phylodynamics (e.g. $n=801$ and $p=2$ continuous coordinates to describe the spread of the West Nile virus [2]).

In these cases with large data size $np$, the matrix-based alternative to BP is prone to numerical inaccuracy and numerical instability in addition to the increased computational time, because it is hard to accurately invert a large matrix. Even when the matrix is of moderate size, numerical inaccuracy can arise when the matrix is ‘ill-conditioned’. These problems were identified under OU models on phylogenetic trees that have closely related sister taxa, or under early burst (EB) models with strong morphological diversification early on during the group radiation, and much slowed-down evolution later on [51–53].

For some simple models, the large $np\times np$ covariance matrix can be decomposed as a Kronecker product of a $p\times p$ trait covariance and a $n\times n$ phylogenetic covariance. This decomposition can simplify the complexity of calculating the likelihood. However, this decomposition is not available under many models, such as the multivariate Brownian motion with shifts in the evolutionary rates (e.g. [54]) or the multivariate Ornstein–Uhlenbeck model with non-scalar rate or selection matrices [21,55].

(c). Belief propagation for continuous traits on trees

To bypass the complexity of matrix inversion, Felsenstein pioneered IC to test for phylogenetic correlation between traits, assuming a BM model on a tree [34]. Many authors then used BP approaches to handle Gaussian models beyond the BM [14,15,17]. Notably, Ho & Ané [16] describe a fast algorithm that can be used for non-Gaussian models as well. Most recently, Mitov et al. [18] highlighted that BP can be applied to a large class of Gaussian models including the BM and the OU process with shifts and variation of rates and selection regimes across branches. Software packages that use these fast BP algorithms include phylolm [16], Rphylopars [17], BEAST [56] or the most recent versions of hOUwie [41] and mvSLOUCH [53].

All the methods cited above only use the first post-order tree traversal of BP to compute the likelihood. A second pre-order traversal allows, in the Gaussian case, for the computation of the distribution of all internal nodes conditionally on the model and on the trait values at the tips. These distributions can then be used for, e.g. ancestral state reconstruction [7], expectation–maximization algorithms for shift detection in the optimal values of an OU [57], or the computation of the gradient of the likelihood in the BM [4,58] or general Gaussian model [59]. Such BP techniques have also been used for taking gradients of the likelihood with respect to branch lengths in sequence evolution models [60,61] or for phylogenetic factor analysis [62,63].

(d). From trees to networks

So far, Felsenstein’s pruning algorithm and related BP approaches have mostly been restricted to phylogenetic trees. There is now ample evidence that reticulation is ubiquitous in all domains of life from biological processes such as lateral gene transfer, hybridization, introgression and gene flow between populations. Networks are recognized to be better than trees for representing the phylogenetic history of species and populations in many groups. Although current studies using networks have few taxa, typically between 10 and 20 (e.g. [12]), they tend to have increasingly more tips as network inference methods become more scalable (e.g. $n=39$ languages in [10]). As viruses are known to be affected by recombination, we also expect future virus studies to use large network phylogenies [64], so that BP will become essential for network studies too. In this work, we describe approaches currently used for trait evolution on phylogenetic networks. We argue that the field of evolutionary biology would benefit from applying BP approaches to networks more systematically. Transferring knowledge from the mature and rich literature on BP would advance evolutionary biology research when phylogenetic networks are used.

(e). Current network approaches for discrete traits

For discrete traits on general networks, very few approaches use BP techniques as far as we know. For DNA data, for example, PhyLiNC [65] and NetRAX [66] extend the typical tree-based model to general networks, assuming no incomplete lineage sorting. That is, each site is assumed to evolve along one of the trees displayed in the network, chosen according to inheritance probabilities at reticulate edges. PhyLiNC assumes independent (unlinked) sites. NetRAX assumes independent loci, which may have a single site each. Each locus may have its own set of branch lengths and substitution model parameters. Both methods calculate the likelihood of a network $N$ via extracting its displayed trees and then applying BP on each tree. Similarly, comparative methods for binary and multi-state traits implemented in PhyloNetworks also extract displayed trees and then apply BP on each displayed tree [67]. While these approaches use BP on each displayed tree, a network with $h$ reticulations can have up to $2^h$ displayed trees. This leads to a computational bottleneck when the number of reticulations increases.

BP approaches have also been used for models with incomplete lineage sorting, modelled by the coalescent [68]. Notably, SNAPP models the evolution of unlinked biallelic markers along a species tree, accounting for incomplete lineage sorting [25]. This method was recently made faster with SNAPPER [69] and extended to phylogenetic networks with SnappNet [70]. The coalescent process introduces the challenge that each site may evolve along any tree, depending on past coalescent events. SNAPP introduced a way to bypass the difficulties of handling coalescent histories and hence decrease computation time. After we describe BP for general graphical models, we recast this innovation as BP on a graphical model formulation of the problem.

BP was also used to calculate the likelihood of the joint sample frequency spectrum (SFS). To account for incomplete lineage sorting on a tree, Kamm et al. [71] use the continuous-time Moran model to reduce computational complexity and assume that each site undergoes at most one mutation. In momi2, Kamm et al. [72] extend the approach to phylogenetic networks by assuming a pulse of admixture at reticulations. The associated graphical model is much simpler than that required by SNAPP or SnappNet, thanks to the assumption of no recurrent mutation.

(f). Current network approaches for continuous traits

Compared with the rich toolkit available for the analysis of continuous traits on trees, the toolkit for phylogenetic networks is still limited. PhyloNetworks includes comparative methods on networks [73], implemented in Julia [74]. These methods extend phylogenetic ANOVA to networks for a continuous response trait predicted by any number of continuous or categorical traits, with residual variation being phylogenetically correlated. So far, the models available in PhyloNetworks include the BM, Pagel’s $\lambda$, possible within-species variation and shifts at reticulations to model transgressive evolution [75,76]. However, all calculations are based on working with the full covariance matrix, without BP. TreeMix [77], ADMIXTOOLS [11,78], poolfstat [79] and AdmixtureBayes [12] use allele frequency as a continuous trait. They model its evolution along a network, or admixture graph, using a Gaussian model in which the evolutionary rate variance is affected by the ancestral allele frequency [80,81]. Again, these methods work with the phylogenetic covariance matrix, rather than BP approaches. They also consider subsets of up to four taxa at a time via $f_2$, $f_3$ and $f_4$ statistics, which simplifies the likelihood calculation. To identify selection and adaptation on a network, PolyGraph [82] and GRoSS [83] assume a similar model and use the full covariance matrix. In summary, BP has yet to be used for continuous trait evolution on networks.

3. Continuous trait evolution on a phylogenetic network

We now present phylogenetic models for the evolution of continuous traits, to which we apply BP later. We generalize the framework in Mitov et al. [18] and Bastide et al. [59] from trees to networks, and we extend the network model in Bastide et al. [75] from the BM to more general evolutionary models. We consider a multivariate $X$ consisting of $p$ continuous traits and model their correlation over time. Our model ignores the potential effects of incomplete lineage sorting on $X$, a reasonable assumption for highly polygenic traits.

(a). Linear Gaussian models

Most random processes used to model continuous trait evolution on a phylogenetic tree are extensions of the BM to capture processes such as evolutionary trends, adaptation and variation in rates across lineages, for example. In its most general form, the linear Gaussian evolutionary model on a tree (referred to as the GLInv family in [18]) assumes that the trait $X_v$ at node $v$ has the following distribution conditional on its parent ${\displaystyle \mathrm{p}\mathrm{a}(\upsilon )}$

$${\displaystyle X_v\mid X_{\text{pa}}(v)\sim \mathcal{N}({\mathit{q}}_v{X}_{\text{pa}}(v)+{\omega }_v,{\mathit{V}}_v)}$$ (3.1)

where the actualization matrix ${\displaystyle {\mathit{q}}_v}$, the trend vector ${\omega }_v$ and the covariance matrix ${\displaystyle {\mathit{V}}_v}$ are appropriately sized and do not depend on trait values ${\displaystyle X_{\mathrm{p}\mathrm{a}}(\upsilon )}$. When the tree is replaced by a network, a node $v$ can have multiple parents ${\displaystyle \mathrm{p}\mathrm{a}(\upsilon )}$. In this case, we can write ${\displaystyle X_{\mathrm{p}\mathrm{a}}(\upsilon )}$ as the vector formed by stacking the elements of ${\displaystyle \{X_u\mid u\in \mathrm{p}\mathrm{a}(\upsilon )\}}$ vertically, with length equal to the number of traits times the number of parents of $v$. In the following, we show that (3.1), already used on trees, can easily be extended to networks to describe both evolutionary models along one lineage and a merging rule at reticulation events.

(b). Evolutionary models along one lineage

For a tree node $v$ with parent node $u$, we need to describe the evolutionary process along one lineage, graphically modelled by the tree edge $e=(u,v)$. It is well known that a wide range of evolutionary models can fit in the general form (3.1) [18,59]. For instance, the BM with variance rate ${\displaystyle \mathbf{\Sigma }}$ (a variance–covariance matrix for a multivariate trait) is described by (3.1) where ${\displaystyle {\mathit{q}}_v}$ is the $p\times p$ identity matrix ${\displaystyle {\mathit{I}}_p}$, there is no trend ${\omega }_v=0$, and the variance is proportional to the edge length $\ell (e)$: ${\displaystyle {\mathit{V}}_v=\ell (e)\mathbf{\Sigma }}$.

Allowing for rate variation amounts to letting the variance rate vary across edges ${\displaystyle \mathbf{\Sigma }=\mathbf{\Sigma }(e)}$. For example, the early burst (EB) model assumes that the variance rate at any given point in the phylogeny depends on the time $t$ from the root to that point, as:

$${\displaystyle \mathbf{\Sigma }(t)={\mathbf{\Sigma }}_0{e}^{bt}.}$$

For this $t$ to be well-defined on a reticulate network, the network needs to be time-consistent (distinct paths from the root to a node all share the same length). The rate $b$ is a rate of variance decay if it is negative, to be expected during adaptive radiations, with a burst of variation near the root (hence early burst) before a slow-down of trait evolution [84]. When $b>0$, this model is called ‘accelerating rate’ (AC) [85]. Clavel & Morlon [86] used a flexible extension of this model (on a tree), replacing $t$ by one or more covariates that are known functions of time, such as the average global temperature and other environmental variables:

$${\displaystyle \mathbf{\Sigma }(t)=\tilde{\sigma }(t,T_1(t),\cdots ,T_k(t)).}$$

Then, the variance accumulated along edge $e=(u,v)$ is given by

$${\displaystyle {\mathit{V}}_v={\int }_{t(u)}^{t(v)}\mathbf{\Sigma }(t)dt.}$$

In the particular case of the EB model, we get

$${\displaystyle {\mathit{V}}_v={\mathbf{\Sigma }}_0{e}^{bt(u)}(e^{b\ell (e)}-1)/b.}$$

Allowing for shifts in the trait value, perhaps due to jumps or cladogenesis, amounts to including ${\omega }_v\ne 0$ for some $v$.

Adaptive evolution is typically modelled by the OU process, which includes a parameter ${\displaystyle {\mathit{A}}_e}$ for the strength of selection along edge $e$. This selection strength is often assumed constant across edges, and is typically denoted as $\alpha$ for a univariate trait. The OU process also includes a primary optimum value ${\displaystyle {\mathit{\theta }}_e}$, which may vary across edges when we are interested in detecting shifts in the adaptive regime across the phylogeny. Under the OU model, the trait evolves along edge $e$ with random drift and a tendency towards ${\displaystyle {\mathit{\theta }}_e}$:

$${\displaystyle d{X}^{(e)}(t)={\mathit{A}}_e({\mathit{\theta }}_e-X^{(e)}(t))dt+{\mathit{R}}_{e}dB(t)}$$

where $B$ is a standard BM and the drift variance is ${\displaystyle {\mathbf{\Sigma }}_e={\mathit{R}}_e{\mathit{R}}_e^{\mathrm{\top }}}$. Then, conditional on the starting value at the start of $e$, the end value $X_v$ is linear Gaussian as in (3.1) with actualization ${\displaystyle {\mathit{q}}_v=e^{-\ell (e){\mathit{A}}_e}}$, trend ${\displaystyle {\omega }_v=(\mathit{I}-e^{-\ell (e){\mathit{A}}_e}){\mathit{\theta }}_e}$ and variance

$${\displaystyle {\mathit{V}}_v={\int }_0^{\ell (e)}{e}^{-s{\mathit{A}}_e}{\mathbf{\Sigma }}_e{e}^{-s{\mathit{A}}_e^{\mathrm{\top }}}ds={\mathit{S}}_e-e^{-\ell (e){\mathit{A}}_e}{\mathit{S}}_e{e}^{-\ell (e){\mathit{A}}_e^{\mathrm{\top }}}}$$

where ${\displaystyle {\mathit{S}}_e}$ is the stationary variance matrix. These equations simplify greatly if ${\displaystyle {\mathit{A}}_e}$ and ${\displaystyle {\mathbf{\Sigma }}_e}$ commute, such as if ${\displaystyle {\mathit{A}}_e}$ is scalar of the form ${\displaystyle {\alpha }_e{\mathit{I}}_p}$, including when the process is univariate. In this case,

$${\displaystyle {\mathit{V}}_v=(1-e^{-2\alpha \ell (e)}){\mathbf{\Sigma }}_e/(2\alpha ).}$$

Shifts in adaptive regimes can be modelled by shifts in any of the parameters ${\displaystyle {\mathit{\theta }}_e}$, ${\displaystyle {\mathit{A}}_e}$ or ${\displaystyle {\mathbf{\Sigma }}_e}$ across edges.

Finally, variation within species, including measurement error, can be easily modelled by grafting one or more edges at each species node to model the fact that the measurement taken from an individual may differ from the true species mean. The model for within-species variation, then, should also follow (3.1) by which an individual value is assumed to be normally distributed with a mean that depends linearly on the species mean, and a variance independent of the species mean—although this variance can vary across species. Most typically, observations from species $v$ are modelled using ${\displaystyle \mathit{q}={\mathit{I}}_p}$, $\omega =0$ and some phenotypic variance to be estimated, that may or may not be tied to the evolutionary variance parameter from the phylogenetic model across species. This additional observation layer can also be used for factor analysis, where the unobserved latent trait evolving on the network has a smaller dimension than the observed traits. In that case, ${\displaystyle \mathit{q}}$ is a rectangular, representing the loading matrix [62,63].

(c). Evolutionary models at reticulations

For a continuous trait and a hybrid node $h$, Bastide et al. [75] and Pickrell & Pritchard [77] assumed that $X_h$ is a weighted average of its immediate parents, using their state immediately before the reticulation event. Specifically, if $h$ has parent edges $e_1,\dots ,e_m$, and if we denote by $X_{\underline{e_k}}$ the state at the end of edge $e_k$ right before the reticulation event ($1\le k\le m$), then the weighted-average model assumes that

$${\displaystyle {\displaystyle X_h=\sum _{e_k\text{ parent of }h}\gamma (e_k)X_{\underset{\_}{e_k}}.}}$$ (3.2)

This model is a reasonable null model for polygenic traits, reflecting the typical observation that hybrid species show intermediate phenotypes. In this model, the biological process underlying the reticulation event (such as gene flow versus hybrid speciation) does not need to be known. Only the proportion of the genome inherited by each parent, $\gamma (e_k)$, needs to be known. Compared with the evolutionary timescale of the phylogeny, the reticulation event is assumed to be instantaneous.

To describe this process as a graphical model, we may add a degree-2 node at the end of each hybrid edge $e$ to store the value $X_{\underline{e}}$, so as to separate the description of the evolutionary process along each edge from the description of the process at a reticulation event. With these extra degree-2 nodes, the weighted-average model (3.2) corresponds to the linear Gaussian model (3.1) with no trend ${\omega }_h=0$, no variance ${\displaystyle {\mathit{V}}_h=\mathbf{0}}$ and with actualization ${\displaystyle {\mathit{q}}_h=[\gamma (e_1){\mathit{I}}_p\dots \gamma (e_m){\mathit{I}}_p]}$ made of scalar diagonal blocks.

Several extensions of this hybrid model can be considered. Bastide et al. [75] modelled transgressive evolution with a shift ${\omega }_h\ne 0$ for the hybrid population to differ from the weighted average of its immediate parents, even possibly taking a value outside their range. Jhwueng & O’Meara [87] considered transgressive shifts at each hybrid node as random variables with a common variance, corresponding to a model with ${\omega }_h=0$ but non-zero variance ${\displaystyle {\mathit{V}}_h}$.

More generally, we may consider models in which the hybrid value is any linear combination of its immediate parents ${\displaystyle {\mathit{q}}_v{X}_{\mathrm{p}\mathrm{a}(\upsilon )}}$ as in (3.1). A biologically relevant model could consider ${\displaystyle {\mathit{q}}_v}$ to be diagonal, with, on the diagonal, parental weights $\gamma (e,j)$ that may depend on the trait $j$ instead of being shared across all $p$ traits.

We may also consider both a fixed transgressive shift ${\omega }_h\ne 0$ and an additional hybrid variance ${\displaystyle {\mathit{V}}_h}$. For both of these components to be identifiable in the typical case when we observe a unique realization of the trait evolution, the model would need extra assumptions to induce sparsity. For example, we may assume that ${\displaystyle {\mathit{V}}_h}$ is shared across all reticulations and is given an informative prior to capture small variations around the parental weighted average. We may also need a sparse model on the set of ${\omega }_h$ parameters, e.g. letting ${\omega }_h\ne 0$ only at a few candidate reticulations $h$, chosen based on external domain knowledge.

For a continuous trait known to be controlled by a single gene, we may prefer a model similar to the discrete trait model presented later in example 4.2, by which $X_h$ takes the value of one of its immediate parent $X_{\underline{e}}$ with probability $\gamma (e)$. This model would no longer be linear Gaussian unless we condition on which parent is being inherited at each reticulation. Such conditioning would reduce the phylogeny to one of its displayed trees. But it would require other techniques to integrate over all parental assignments to each hybrid population, such as Markov Chain Monte Carlo or expectation-maximization.

(d). Evolutionary models with interacting populations

Models have been proposed in which the evolution of $X^{(e)}(t)$ along one edge $e$ depends on the state on other edges existing at the same time $t$ [88–91]. These models can describe ‘phenotype matching’ that may arise from ecological interactions (mutualism, competition) or demographic interactions (migration), in which traits across species or populations converge to or diverge from one another. To express this coevolution, we consider the set $E(t)$ of edges contemporary to one another at time $t$ and divide the phylogeny into epochs: time intervals $[{\tau }_i,{\tau }_{i+1}]$ during which the set $E(t)$ of interacting lineages is constant, denoted as $E_i$. Within each epoch $i$ (i.e. $t\in [{\tau }_i,{\tau }_{i+1}]$), the vector of all traits $(X^{(e)}(t){)}_{e\in E_i}$ is modelled by a linear stochastic differential equation. Since its mean is linear in and its variance independent of the starting value $(X^{(e)}({\tau }_i){)}_{e\in E_i}$, these models are linear Gaussian [89,90]. In fact, they can be expressed by (3.1) on a supergraph of the original phylogeny, in which an edge $(u,v)$ is added if $u$ is at the start ${\tau }_i$ of some epoch $i$, $v$ is at the end ${\tau }_{i+1}$, and if the mean of $X_v$ conditional on all traits at time ${\tau }_i$ has a non-zero coefficient for $X_u$. The specific form of ${\displaystyle {\mathit{q}}_v}$, ${\omega }_v$ and ${\displaystyle {\mathit{V}}_v}$ in (3.1) depends on the specific interaction model, and may be more complex than the merging rule (3.2).

4. A short review of graphical models and belief propagation

Implementing BP techniques on general networks is more complex than on trees and involves the construction of an auxiliary graph known as a clique tree or cluster graph (§4b). To explain why, we review here the main ideas of graphical models and belief propagation for likelihood calculation.

(a). Graphical models

A probabilistic graphical model is a graph representation of a probability distribution. Each node in the graph represents a random variable, typically univariate but possibly multivariate. We focus here on graphical models with directed edges on a directed acyclic graph (DAG). Edges represent dependencies between variables, where the direction is typically used to represent causation. The graph expresses conditional independencies satisfied by the joint distribution of all the variables at all nodes in the graph.

Given the directional nature of evolution and inheritance, models for trait evolution on a phylogeny are often readily formulated as directed graphical models. Höhna et al. [92] demonstrate the utility of representing phylogenetic models as graphical models for exposing assumptions and for interpretation and implementation. They present a range of examples common in evolutionary biology, with a focus on how graphical models facilitate greater modularity and transparency. Directed graphical models have also been used to parametrize distributions on tree topologies for accurate approximations of posterior distributions and for variational inference [93–95]. In this issue, similar DAGs are used to store phylogenetic trees efficiently, for parsimony-based inference [96]. Here we focus instead on the computational gains that BP allows in graphical models.

A directed graphical model consists of a DAG $G$ and a set of conditional distributions, one for each node in $G$. At a node $v$ with parent nodes ${\displaystyle \mathrm{p}\mathrm{a}(\upsilon )}$, the distribution of variable $X_v$ conditional on its set of parent variables ${\displaystyle X_{\mathrm{p}\mathrm{a}(\upsilon )}=\{X_u;u\in \mathrm{p}\mathrm{a}(\upsilon )\}}$ is given by a factor ${\varphi }_v$, which is a function whose scope is the set of variables from $v$ and ${\displaystyle \mathrm{p}\mathrm{a}(\upsilon )}$. For each node $v$, the set formed by this node and its parents ${\displaystyle \{v\}\cup \mathrm{p}\mathrm{a}(\upsilon )}$ is called a node family. If $V$ denotes the vertex set of $G$, then the set of factors $\{{\varphi }_v,v\in V\}$ defines the joint density of the graphical model as

$${\displaystyle p_{\theta }(X_v;v\in V)=\mathbf{\prod }_{v\in V}{\varphi }_v\left(X_v|X_u,\theta ;u\in \mathrm{p}\mathrm{a}(v)\right)}$$ (4.1)

where we add the possible dependence of factors on model parameters $\theta$. This factor formulation implies that, conditional on its parents, $X_v$ is independent of any non-descendant node (e.g. ‘grandparents’) [26].

Example 4.1. Brownian motion (BM) on a tree. Consider the phylogenetic tree $T$ in figure 1a. The graphical model for the node states of $T$ under a BM, whose parameters $\theta$ are the trait evolutionary variance rate ${\sigma }^2$, the ancestral state at the root $x_{\rho }$ and edge lengths ${\ell }_i$, has the same topology as $T$. On a tree, each node family consists of a node $v$ and its single parent, or the root $\rho$ by itself. The distribution ${\varphi }_{\rho }$ may be deterministic as when $x_{\rho }$ is a fixed parameter of the model, or it may be given a prior distribution ${\varphi }_{\rho }$.

Figure 1.

Example graphical model on a phylogenetic tree with factors defined by the BM. The joint distribution of all variables at all nodes is given by the product of factors ${\prod }_v{\varphi }_v$ as in (4.1), where ${\varphi }_v$ is the distribution of $x_v$ conditional on its parent variable ${\displaystyle x_{\mathrm{p}\mathrm{a}(\upsilon )}}$: $\mathcal{N}(x_{\mathrm{p}\mathrm{a}(\upsilon )},{\sigma }^2{\ell }_v)$ under the BM. (a) Phylogenetic tree $T$. The graphical model uses the same graph $G=T$. (b) Clique tree $\mathcal{U}$ for the graphical model (see definition 4.1 in §4b(i)). Its nodes are clusters of variables in $T$ (ellipses). Each edge is labelled by a sepset (squares, see definition 4.1 (ii)): a subset of variables shared by adjacent clusters.

Example 4.2. Discrete trait on a network. A rooted phylogenetic network is a DAG with a single root and taxon-labelled leaves (or tips). A node with at most one parent is called a tree node and its incoming edge is a tree edge. A node with multiple parents is called a hybrid node and represents a population (or species more generally) with mixed ancestry. An edge ${\displaystyle e=(u,h)}$ going into a hybrid node ${\displaystyle h}$ is called a hybrid edge. It is assigned an inheritance probability ${\displaystyle \gamma (e)>0}$ that represents the proportion of the genome in $h$ that was inherited from the parent population $u$ (via edge $e$). Obviously, at each hybrid node $h$ we must have ${\displaystyle {\sum }_{u\in \mathrm{p}\mathrm{a}(h)}\gamma ((u,h))=1}$. The phylogenetic network $N$ in figure 2a has one hybrid node $x_5$ whose genetic makeup comes from $x_4$ with proportion $0.4$ and from $x_6$ with proportion $0.6$.

Figure 2.

(a) Phylogenetic network ${\displaystyle N}$ with hybrid edges shown in blue and annotated with γ values. The graphical model uses the same graph ${\displaystyle G=N}$. ${\displaystyle N}$ displays two trees, depending on which hybrid edge is retained. One tree, with sister taxa 1 and 2, has probability $0.4$. The other tree, with sister taxa 2 and 3, is displayed with probability $0.6$. The distribution of the hybrid node $x_5$ depends on both its parents and induces a factor cluster $\{x_4,x_5,x_6\}$ of size 3 in $\mathcal{U}$ and ${\mathcal{U}}^{*}$ (see the family-preserving property in definition 4.1 (i)). (b) Clique tree $\mathcal{U}$ for the graphical model. (c) Cluster graph ${\mathcal{U}}^{*}$ (see definition 4.1) for the same graphical model in which $\{x_4,x_6,x_{\rho }\}$ in $\mathcal{U}$ is replaced by smaller clusters $\{x_4,x_{\rho }\}$, $\{x_6,x_{\rho }\}$ and $\{x_{\rho }\}$ that induce a cycle. Leaf clusters are not shown.

For a discrete trait $X$, the traditional model of evolution on a tree can be extended to a network $N$ as follows. Along each edge $e$, $X$ evolves according to a Markov process with some transition rate matrix $Q$ for an amount of time $\ell (e)$ that depends on the edge. At a tree node, the state of $X$ at the end of its parent edge is passed as the starting value to each daughter lineage, as in the traditional tree model. At reticulations, we follow previous authors to model the value $x_h$ at a hybrid node $h$ [65–67,97]. Let $x_e$ denote the state at the end of edge $e$, going forward in time. If $h$ has $m$ parent edges $e_1,\cdots ,e_m$, then $x_h$ is assumed to take value $x_{e_k}$ with probability $\gamma (e_k)$. This model reflects the idea that the trait is controlled by unknown genes, but the proportion of genes inherited from each parent is known. Incomplete lineage sorting, which can lead to hemiplasy for a trait [98], is unaccounted for. Similar to Example 4.1, the graphical model uses the topology of the network $N$.

To describe the factors of this graphical model and simplify notations, consider the case when $X$ is binary with states 0 and 1. For a tree node $v$, the factor ${\varphi }_v$ can be represented by the $2\times 2$ matrix $\exp (\ell (e)Q)$, where $e$ is the parent edge of $v$. For a hybrid node $h$ with $m$ parents $p_1,\cdots ,p_m$ and edges $e_k=(p_k,h)$ with $\gamma (e_k)={\gamma }_k$, the factor ${\varphi }_h$ has scope $(X_h,X_{p_1},\cdots ,X_{p_m})$ and can be described by a $2\times 2^m$ matrix to store the conditional probabilities $\mathbb{P}(X_h=j|X_{p_1}=i_1,\cdots ,X_{p_m}=i_m)$. This is a $2\times 4$ matrix in the typical case when $h$ is admixed from $m=2$ parental populations. With $m=2$ and with parental values $(X_{p_1},X_{p_2})$ arranged in ordered $((\mathrm{0,0}),(\mathrm{0,1}),(\mathrm{1,0}),(\mathrm{1,1}))$, then

$${\displaystyle {\varphi }_h=\left(\begin{array}{cccc}1& {\gamma }_1& {\gamma }_2& 0\\ 0& {\gamma }_2& {\gamma }_1& 1\end{array}\right).}$$

Example 4.3. Binary trait with incomplete lineage sorting (ILS). More complex evolutionary processes, such as ILS, can result in a graph $G$ for the graphical model that is constructed from but not identical to the phylogeny. Such is the case for the evolution of a genetic marker whose gene tree is generated according to the coalescent model along the species phylogeny [68,99,100]. For a marker with two alleles, say ‘green’ and ‘red’, the data consist of the number of red alleles in a sample of individuals from each species. In electronic supplementary material, §A, we formulate the likelihood calculations by Bryant et al. [25] on a species tree and by Rabier et al. [70] on a species network as belief propagation. For this evolutionary model, the graph $G$ of the associated graphical model differs significantly from the original phylogeny (illustrated in electronic supplementary material, figures S1 and S2).

Example 4.4. Discrete trait determined by an unobserved continuous trait. The threshold model uses a latent (unobserved) continuous trait, or ‘liability’, evolving as a Brownian motion to determine an observed discrete trait. The discrete trait changes state when the liability crosses a threshold [23,101]. As the liability is unobserved, the discrete trait has ‘memory’ and is not Markovian [24]: the probability to transition from one state to another depends on the amount of time spent in the current state. However, we can express the model as a graphical model suitable for BP by modelling the liability at all nodes in the phylogeny, and by adding an observation layer to the graph $G$ for the value of the discrete trait at each tip. This layer adds a pendant edge to connect an observed trait to its corresponding liability. Thresholding adds significant complexity to likelihood calculations. Existing algorithms, on trees, use approximations [24,102] or resort to sampling the latent liability in a Bayesian context [3,6,103], including Hamiltonian techniques that exploit the gradient [4,5].

Representing discrete traits as thresholded liabilities makes it easy to model correlations between continuous and discrete traits in multivariate datasets with both types of traits [3]. Using graphical models, such models can leverage BP techniques and hence be extended to phylogenetic networks and general Gaussian processes.

(b). Belief propagation

BP is a framework for efficiently computing various integrals of the factored density $p_{\theta }$ by grouping nodes and their associated variables into clusters and integrating them out according to rules along a clique tree (also known as a junction tree, join tree or tree decomposition) or along a cluster graph, more generally.

(i). Cluster graphs and clique trees

Definition 4.1 (cluster graph and clique tree). Let $\mathrm{\Phi }=\{{\varphi }_v,v\in V\}$ be the factors of a graphical model on graph $G$ and let $\mathcal{U}=(\mathcal{V},\mathcal{E})$ be an undirected graph whose nodes ${\mathcal{C}}_i\in \mathcal{V}$, called clusters, are sets of variables in the scope of $\mathrm{\Phi }$. $\mathcal{U}$ is a cluster graph for $\mathrm{\Phi }$ if it satisfies the following properties:

1. (family-preserving) There exists a map $\alpha :\mathrm{\Phi }\to \mathcal{V}$ such that for each factor ${\varphi }_v$, its scope (node family for node $v$ in the graphical model) is a subset of the cluster $\alpha ({\varphi }_v)$.

2. (edge-labelled) Each edge $\{{\mathcal{C}}_i,{\mathcal{C}}_j\}$ in $\mathcal{E}$ is labelled with a non-empty sepset ${\mathcal{S}}_{i,j}$ (‘separating set’) such that ${\mathcal{S}}_{i,j}\subseteq {\mathcal{C}}_i\cap {\mathcal{C}}_j$.

3. (running intersection) For each variable $x$ in the scope of $\mathrm{\Phi }$, ${\mathcal{E}}_x\subseteq \mathcal{E}$, the set of edges with $x$ in their sepsets forms a tree that spans ${\mathcal{V}}_x\subseteq \mathcal{V}$, the set of clusters that contain $x$.

If $\mathcal{U}$ is acyclic, then $\mathcal{U}$ is called a clique tree and we refer to its nodes as cliques. In this case, properties (ii) and (iii) imply that ${\mathcal{S}}_{i,j}={\mathcal{C}}_i\cap {\mathcal{C}}_j$.

A clique tree $\mathcal{U}$ is shown in figure 1b for the BM model from example 4.1, on the tree $T$ in figure 1a. To check the running intersection property for $x_5$, for example, we extract the graph defined by edges with $x_5$ in their sepsets (squares). There are two such edges. They induce a subtree of $\mathcal{U}$ that connects all three clusters (ellipses) containing $x_5$, as desired. More generally, when the graphical model is defined on a tree $T$, a corresponding clique tree $\mathcal{U}$ is easily constructed, where cliques in $\mathcal{U}$ correspond to edges in $T$ and edges in $\mathcal{U}$ correspond to nodes in $T$. Multiple clique trees can be constructed for a given graphical model. In this example, the clique $\{x_{\rho }\}$ (shown at the top) could be suppressed because it is a subset of adjacent cliques.

For the network $N$ in figure 2a and the evolution of a discrete trait in example 4.2, one possible clique tree $\mathcal{U}$ is shown in figure 2b. Note that $x_5,x_4$ and $x_6$ have to appear together in at least one of the clusters for the clique tree to be family-preserving (property (i)), because $x_4$ and $x_6$ are partners with a common child $x_5$ whose distribution depends on both of their states.

We first focus on clique trees, which provide a structure for the exact likelihood calculation. In §5, we discuss the advantages of cluster graphs to approximate the likelihood at a lower computational cost.

(ii). Evidence

To calculate the likelihood of the data, or the marginal distribution of the traits at some node conditional on the data, we inject evidence into the model in one of two equivalent ways. For each observed value ${\mathrm{x}}_{v,t}$ of the $t^{\mathrm{th}}$ trait $x_{v,t}$ at node $v$, we add to the model the indicator function ${\displaystyle {\mathbb{1}}_{\{{\mathrm{x}}_{v,t}\}}(x_{v,t})}$ as an additional factor. Equivalently, we can plug in the observed value ${\mathrm{x}}_{v,t}$ in place of the variable $x_{v,t}$ in all factors where $x_{v,t}$ appears, and then drop $x_{v,t}$ from the scope of all these factors. This second approach is more tractable than the first to avoid the degenerate zero-variance Dirac distribution. But it requires careful bookkeeping of the scope and of re-parametrization of each factor with missing data when some traits, but not all, are observed at some nodes. Below, we assume that the factors and their scopes have been modified to absorb evidence from the data.

(iii). Belief update message passing

There are multiple equivalent algorithms to perform BP. We focus here on the belief update algorithm. It assigns a belief to each cluster and to each sepset in the cluster graph. After running the algorithm, each belief should provide the marginal probability of the variables in its scope and of the observed data, with all other variables integrated out as desired to calculate the likelihood. The belief of cluster ${\mathcal{C}}_i$, denoted as ${\beta }_i$, is initialized as the product of all factors assigned to that cluster:

$${\displaystyle {\beta }_i^{(\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l})}={\psi }_i=\prod _{\varphi ;\alpha (\varphi )={\mathcal{C}}_i}\varphi \text{ for cluster }{\mathcal{C}}_i}$$ (4.2)

The belief of an edge between cluster $i$ and $j$, denoted as ${\mu }_{i,j}$, is initialized to the constant function 1. These beliefs are then updated iteratively by passing messages. Passing a message from ${\mathcal{C}}_i$ to ${\mathcal{C}}_j$ along an edge with sepset ${\mathcal{S}}_{i,j}$ corresponds to passing information about the marginal distribution of the variables in ${\mathcal{S}}_{i,j}$ as shown in algorithm 1. If $\mathcal{U}$ is a clique tree, then all beliefs converge to the true marginal probability of their variables and of the observed data, after traversing $\mathcal{U}$ only twice: once to pass messages from leaf cliques towards some root clique, and then back from the root clique to the leaf cliques. If our goal is to calculate the likelihood, then one traversal is sufficient. Once the root clique has received messages from all its neighbouring cliques, we can marginalize over all its variables (similar to step 1) to obtain the probability of the observed data only, which is the likelihood. The second traversal is necessary to obtain the marginal probability of all variables, such as if one is interested in the posterior distribution of ancestral states conditional on the observed data.

Some equivalent formulations of BP only store sepset messages, and avoid storing cluster beliefs. This strategy requires less memory but more computing time if $\mathcal{U}$ is traversed multiple times.

In example 4.1 on a tree (figure 1a), the conditional distribution of $x_v$ at a non-root node $v$ corresponds to a factor ${\varphi }_v$ for the BM model along edge ${\displaystyle (\mathrm{p}\mathrm{a}(v),v)}$ in $T$. This factor is assigned to clique ${\mathcal{C}}_v=\{\mathrm{p}\mathrm{a}(v),v\}$ in $\mathcal{U}$ to initialize the belief ${\beta }_v$ of ${\mathcal{C}}_v$. If $v$ is a leaf in $T$, then ${\beta }_v$ is further multiplied by the indicator function at the value ${\mathrm{x}}_v$ observed at $v$, such that the belief of clique ${\mathcal{C}}_v$ can be expressed as a function of the leaf’s parent state only: ${\displaystyle {\varphi }_v(x_{\mathrm{p}\mathrm{a}(v)})=\mathbb{P}({\mathrm{x}}_v\mid x_{\mathrm{p}\mathrm{a}(v)})}$. The prior distribution $\varphi (x_{\rho })$ at the root $\rho$ of $T$ (which can be an indicator function if the root value is fixed as a model parameter) can be assigned to any clique containing $\rho$. In figure 1, $\mathcal{U}$ includes a clique ${\mathcal{C}}_{\rho }=\{x_{\rho }\}$ drawn at the top, to which we assign the root prior ${\varphi }_{\rho }(x_{\rho })$ and which we will use as the root of $\mathcal{U}$. Since $\mathcal{U}$ is a clique tree, BP converges after traversing $\mathcal{U}$ twice: from the tips to ${\mathcal{C}}_{\rho }$ and then back to the tips. IC [27,34] implements the first ‘rootwards’ traversal of BP. For example, the belief of clique $\{x_5,x_{\rho }\}$ after receiving messages (steps 1−3) from both of its daughter cliques is the function

$${\beta }_5(x_5,x_{\rho })=\exp (-\frac{(x_{\rho }-x_5{)}^2}{2{\ell }_5}-\frac{(x_5-{\mathrm{x}}_5^{*}{)}^2}{2v_5^{*}}+g_5^{*})$$

where

$${\displaystyle {\mathrm{x}}_5^{\ast }=\frac{{\ell }_2{\mathrm{x}}_1+{\ell }_1{\mathrm{x}}_2}{{\ell }_1+{\ell }_2},v_5^{\ast }=\frac{{\ell }_1{\ell }_2}{{\ell }_1+{\ell }_2}\text{and}{g}_5^{\ast }=-\frac{({\mathrm{x}}_2-{\mathrm{x}}_1{)}^2}{2({\ell }_1+{\ell }_2)}-\log ((2\pi {)}^{3/2}{\ell }_1{\ell }_2{\ell }_5)}$$

are quantities calculated for IC: ${\mathrm{x}}_5^{*}$ corresponds to the estimated ancestral state at node 5, $v_5^{*}$ corresponds to the extra length added to ${\ell }_5$ when pruning the daughters of node 5 and $g_5^{*}$ captures the contrast $({\mathrm{x}}_2-{\mathrm{x}}_1)/\sqrt{{\ell }_1+{\ell }_2}$ below node 5. At this stage of BP, ${\beta }_5(x_5,x_{\rho })$ can be interpreted as $\mathbb{P}({\mathrm{x}}_1,{\mathrm{x}}_2,x_5|x_{\rho })$ such that the message ${\stackrel{\mathrm{~}}{\mu }}_{5\to \rho }(x_{\rho })$ sent from $\{x_5,x_{\rho }\}$ to the root clique ${\mathcal{C}}_{\rho }$ is the partial likelihood $\mathbb{P}({\mathrm{x}}_1,{\mathrm{x}}_2|x_{\rho })$ after $x_5$ is integrated out. The first pass is complete when ${\mathcal{C}}_{\rho }$ has received messages from all its neighbours. Its final belief is then ${\beta }_{\rho }(x_{\rho })=\mathbb{P}({\mathrm{x}}_1,\cdots ,{\mathrm{x}}_4|x_{\rho }){\varphi }_{\rho }(x_{\rho })$. If $x_{\rho }$ is a fixed model parameter, then this is the likelihood. Otherwise, we get the likelihood by integrating out $x_{\rho }$ in ${\beta }_{\rho }(x_{\rho })$.

In example 4.2 on a network (figure 2), we label the cliques in $\mathcal{U}$ as follows: ${\mathcal{C}}_v=\{x_v,x_{\mathrm{p}\mathrm{a}(v)}\}$ for leaves $v=\mathrm{1,2,3}$, ${\mathcal{C}}_5=\{x_5,x_4,x_6\}$ for hybrid node $v=5$ and its parents, and ${\mathcal{C}}_{\rho }=\{x_4,x_6,x_{\rho }\}$. To initialize beliefs, we assign ${\varphi }_v$ to ${\mathcal{C}}_v$ for $v=\mathrm{1,2,3,5}$, and ${\varphi }_4$, ${\varphi }_6$ are both assigned to ${\mathcal{C}}_{\rho }$. Unlike in example 4.1, a clique may correspond to more than a single edge in $N$. This is expected at a hybrid node $h$, because the factor describing its conditional distribution needs to contain $h$ and both of its parents. But for $\mathcal{U}$ to be a clique tree, the root clique ${\mathcal{C}}_{\rho }$ also has to contain the factors from two edges in $N$. Also, unlike for trees, sepsets may contain more than a single node. Here, the two large cliques are separated by $\{x_4,x_6\}$ so they will send messages $\stackrel{\mathrm{~}}{\mu }(x_4,x_6)$ about the joint distribution of these two variables. In this binary trait setting, these messages and sepset beliefs can be stored as $2\times 2$ arrays, and the three-node clique beliefs can be stored as arrays of $2^3$ values. As they involve more variables than when $G$ is a tree (in which case BP would store only two values at each sepset), storing and updating them requires more computing time and memory.

More generally, we see that the computational complexity of BP scales with the size of the cliques and sepsets. This complexity may become prohibitive on a more complex phylogenetic network, even for a simple binary trait without ILS, if the size of the largest cluster in $\mathcal{U}$ is too large—a topic that we explore later.

Example 4.3 illustrates the fact that beliefs cannot always be interpreted as partial (or full) likelihoods at every step of BP, unlike in examples 4.1 and 4.2. For example, consider the tip clique ${\mathcal{C}}_1$ containing the total number of alleles and the number of red alleles in species 1, and the number of their ancestral alleles ($n$ total, of which $r$ are red) just before the speciation event that led to species 1 (electronic supplementary material, figure S1(c). At the first iteration of BP, the first message sent by ${\mathcal{C}}_1$ is the quantity denoted by $F^T(n,r)$ in Bryant et al. [25]. It is not a partial likelihood because it is not the likelihood of some partial subset of the data conditional on some ancestral values ($n$ and $r$). Intuitively, this is because nodes with data below variables in ${\mathcal{C}}_1$ in $G$ are not all below ${\mathcal{C}}_1$ in the clique tree. Information from these data will flow towards ${\mathcal{C}}_1$ at later steps. The beauty of BP is that after a second traversal of the clique tree, ${\mathcal{C}}_1$’s belief is guaranteed to converge to the likelihood of the full data, conditional on the state of the clique variables. See electronic supplementary material, §A.1, for details.

(iv). Clique tree construction

For a given graphical model on $G$, there are many possible clique trees and cluster graphs. For running BP, it is advantageous to have small clusters and small sepsets. Indeed, clusters and sepsets with fewer variables require less memory to store beliefs and less computing time to run steps 1 (integration) and 2 (belief update). Ideally, we would like to find the best clique tree, whose largest clique is of the smallest size possible. For a general graph $G$, finding this best clique tree is hard but good heuristics exist [26].

The first step is to create the moralized graph $G^m$ from $G$. This is done by connecting all nodes that share a common child, and then undirecting all edges. We can then triangulate $G^m$, that is, build a new graph $H$ by adding edges to $G^m$ such that $H$ is chordal (any cycle includes a chord). The width of $H$ is the size of its largest clique minus 1. The treewidth of $G^m$ is the smallest width of all its possible triangulations $H$. Finding $H$ of minimum width is hard, though efficient heuristics exist (e.g. greedy minimum-fill [104,105]). The nodes of $\mathcal{U}$ are then defined as the maximal cliques of $H$ [106]. Finally, the edges of $\mathcal{U}$ are formed such that $\mathcal{U}$ becomes a tree and such that the sum of the sepset sizes is maximum, by finding a maximum spanning tree using Kruskal’s algorithm or Prim’s algorithm [107]. All these steps have polynomial complexity.

(c). Belief propagation for Gaussian models

Before discussing BP on cluster graphs that are not clique trees, we focus on BP updates for the evolutionary models presented in §3. On a phylogenetic network $N$, the joint distribution of all present and ancestral species $(X_v{)}_{v\in N}$ is multivariate Gaussian precisely when it comes from a graphical model on $N$ whose factors ${\varphi }_v$ are linear Gaussian [26]. The factor at node $v$ is linear Gaussian if, conditional on its parents, $X_v$ is Gaussian with a mean that is linear in the parental values and a variance independent of parental values, hence the term ${\mathcal{G}}_{\text{LInv}}$ used by Moran et al. [18]. In other words, for the joint process to be Gaussian, each factor ${\displaystyle {\varphi }_v(x_v\mid x_{\mathrm{p}\mathrm{a}(\upsilon )})}$ should be of the form (3.1).

Such models have been called Gaussian Bayesian networks or graphical Gaussian networks, and are special cases of Gaussian processes (on a graph). These Gaussian models are convenient for BP because linear Gaussian factors have a convenient parametrization that allows for a compact representation of beliefs and belief update operations. Namely, the factor giving the conditional distribution ${\displaystyle {\varphi }_v(x_v\mid x_{\mathrm{p}\mathrm{a}(\upsilon )})}$ from (3.1) can be expressed in a canonical form as the exponential of a quadratic form:

$${\displaystyle \text{C}(x;\mathit{K},h,g)=\exp \left(-\frac{1}{2}{x}^{\mathrm{\top }}\mathit{K}x+h^{\mathrm{\top }}x+g\right).}$$ (4.3)

For example, if we think of ${\displaystyle {\varphi }_v(x_v\mid x_{\mathrm{p}\mathrm{a}(\upsilon )})}$ as a function of $x_v$ primarily, we may use the parametrization ${\displaystyle \text{C}(x_v;\mathit{K},h,g)}$ with

$${\displaystyle \mathit{K}={\mathit{V}}_v^{-1},h={\mathit{V}}_v^{-1}\left({\mathit{q}}_v{x}_{\mathrm{p}\mathrm{a}(\upsilon )}+{\omega }_v\right)\text{ and }g=-\frac{1}{2}\left(\log |2\pi {\mathit{V}}_v|+\Vert {\mathit{q}}_v{x}_{\mathrm{p}\mathrm{a}(\upsilon )}+{\omega }_v{\Vert }_{{\mathit{V}}_v^{-1}}^2\right)}$$

where ${\displaystyle \Vert y{\Vert }_{\mathit{M}}^2}$ denotes ${\displaystyle y^{\mathrm{\top }}\mathit{M}y}$. We can also express ${\varphi }_v$ as a canonical form over its full scope

$${\displaystyle {\varphi }_v(x_v\mid x_{\mathrm{p}\mathrm{a}(\upsilon )})=\text{C}\left(\left[\begin{array}{c}{x}_v\\ x_{\mathrm{p}\mathrm{a}(\upsilon )}\end{array}\right];{\mathit{K}}_v,h_v,g_v\right)}$$

with

$${\displaystyle \begin{array}{r}{\mathit{K}}_v=\left[\begin{array}{cc}{\mathit{V}}_v^{-1}& -{\mathit{V}}_v^{-1}{\mathit{q}}_v\\ -{\mathit{q}}_v^{\mathrm{\top }}{\mathit{V}}_v^{-1}& {\mathit{q}}_v^{\mathrm{\top }}{\mathit{V}}_v^{-1}{\mathit{q}}_v\end{array}\right]=\left[\begin{array}{c}\mathit{I}\\ -{\mathit{q}}_v^{\mathrm{\top }}\end{array}\right]{\mathit{V}}_v^{-1}\left[\begin{array}{cc}\mathit{I}& -{\mathit{q}}_v\end{array}\right],h_v=\left[\begin{array}{c}{\mathit{V}}_v^{-1}{\omega }_v\\ -{\mathit{q}}_v^{\mathrm{\top }}{\mathit{V}}_v^{-1}{\omega }_v\end{array}\right],g_v=-\frac{1}{2}(\log |2\pi {\mathit{V}}_v|+\Vert {\omega }_v{\Vert }_{{\mathit{V}}_v^{-1}}).\end{array}}$$ (4.4)

If $v$ is a leaf with fully observed data, then we need to plug-in the data ${\mathrm{x}}_v$ into ${\varphi }_v$ and consider this factor as a function of ${\displaystyle x_{\mathrm{p}\mathrm{a}(\upsilon )}}$ only. We can express ${\displaystyle {\varphi }_v({\mathrm{x}}_v\mid x_{\mathrm{p}\mathrm{a}(\upsilon )})}$ as the canonical form ${\displaystyle \text{C}(x_{\mathrm{p}\mathrm{a}(\upsilon )};\mathit{K},h,g)}$ with

$${\displaystyle \mathit{K}={\mathit{q}}_v^{\mathrm{\top }}{\mathit{V}}_v^{-1}{\mathit{q}}_v,h={\mathit{q}}_v^{\mathrm{\top }}{\mathit{V}}_v^{-1}({\mathrm{x}}_v-{\omega }_v)\text{ and }g=-\frac{1}{2}\left(\log |2\pi {\mathit{V}}_v|+\Vert {\mathrm{x}}_v-{\omega }_v{\Vert }_{{\mathit{V}}_v^{-1}}^2\right).}$$

If data are partially observed at leaf $v$, the same principle applies. We can plug-in the observed traits into ${\varphi }_v$ and express ${\varphi }_v$ as a canonical form over its reduced scope: ${\displaystyle x_{\mathrm{p}\mathrm{a}(\upsilon )}}$ and any unobserved $x_{v,t}$. Some quadratic terms captured by ${\displaystyle {\mathit{K}}_v}$ on the full scope become linear or constant terms after plugging-in the data, and some linear terms captured by $h_v$ on the full scope become constant terms in the canonical form on the reduced scope.

An important property of this canonical form is its closure under the belief update operations: marginalization (step 1) and factor product (step 2). Indeed, the product of two canonical forms with the same scope satisfies

$${\displaystyle \text{C}(x;{\mathit{K}}_1,h_1,g_1)\text{C}(x;{\mathit{K}}_2,h_2,g_2)=\text{C}(x;{\mathit{K}}_1+{\mathit{K}}_2,h_1+h_2,g_1+g_2).}$$

Now consider marginalizing a factor ${\displaystyle \text{C}(x;\mathit{K},h,g)}$ to a subvector $x^{*}$ of $x$, by integrating out the elements $x\setminus x^{*}$ of $x$ . Let ${\displaystyle {\mathit{K}}_{\mathrm{S}}}$ and ${\displaystyle {\mathit{K}}_{\mathrm{I}}}$ be the submatrices of ${\displaystyle \mathit{K}}$ that correspond to $x^{*}$ (Scope of marginal or Sepset) and $x\setminus x^{*}$ (variables to be Integrated out), and let ${\displaystyle {\mathit{K}}_{\mathrm{S},\mathrm{I}}={\mathit{K}}_{\mathrm{I},\mathrm{S}}^{\mathrm{\top }}}$ be the cross-terms. If ${\displaystyle {\mathit{K}}_{\mathrm{I}}}$ is invertible, then:

$${\displaystyle \int \text{C}(x;\mathit{K},h,g)d(x\setminus x^{\ast })=\text{C}(x^{\ast };{\mathit{K}}^{\ast },h^{\ast },g^{\ast })}$$

where ${\displaystyle {\mathit{K}}^{\ast }={\mathit{K}}_{\mathrm{S}}-{\mathit{K}}_{\mathrm{S},\mathrm{I}}{\mathit{K}}_{\mathrm{I}}^{-1}{\mathit{K}}_{\mathrm{I},\mathrm{S}}}$, ${\displaystyle h^{\ast }=h_{\mathrm{S}}-{\mathit{K}}_{\mathrm{S},\mathrm{I}}{\mathit{K}}_{\mathrm{I}}^{-1}{h}_{\mathrm{I}}}$ with $h_{\mathrm{S}}$ and $h_{\mathrm{I}}$ defined as the subvector of $h$ corresponding to $x^{*}$ and $x\setminus x^{*}$ respectively, and ${\displaystyle g^{\ast }=g+(\log |2\pi {\mathit{K}}_{\mathrm{I}}^{-1}|+\Vert h_{\mathrm{I}}{\Vert }_{{\mathit{K}}_{\mathrm{I}}^{-1}})/2}$.

If the factors of a Gaussian network are non-deterministic, then each belief can be parametrized by its canonical form, and the above equations can be applied to update the cluster and sepset beliefs for BP (algorithm 1). For cluster ${\mathcal{C}}_i$, let ${\displaystyle ({\mathit{K}}_i,h_i,g_i)}$ parametrize its belief ${\beta }_i$. For sepset ${\mathcal{S}}_{i,j}$, let ${\displaystyle ({\mathit{K}}_{i,j},h_{i,j},g_{i,j})}$ parametrize its belief ${\mu }_{i,j}$. Also, for step 1 of BP, let ${\displaystyle ({\mathit{K}}_{i\to j},h_{i\to j},g_{i\to j})}$ parametrize the message ${\stackrel{\mathrm{~}}{\mu }}_{i\to j}$ sent from ${\mathcal{C}}_i$ to ${\mathcal{C}}_j$. Then BP updates can be expressed as shown below.

In step 1, ${\displaystyle {\mathit{K}}_{\mathrm{S}}}$ and ${\displaystyle {\mathit{K}}_{\mathrm{I}}}$ are the submatrices of ${\displaystyle {\mathit{K}}_i}$ that correspond to ${\mathcal{S}}_{i,j}$ and ${\mathcal{C}}_i\setminus {\mathcal{S}}_{i,j}$. Similarly, $h_{\mathrm{S}}$ and $h_{\mathrm{I}}$ are subvectors of $h_i$. In step 2, ${\displaystyle \text{ext}({\mathit{K}}_{\tilde{\mu }}-{\mathit{K}}_{i,j})}$ extends ${\displaystyle {\mathit{K}}_{\tilde{\mu }}-{\mathit{K}}_{i,j}}$ to the same scope as ${\displaystyle {\mathit{K}}_j}$ by padding it with zero rows and zero columns for ${\mathcal{C}}_j\setminus {\mathcal{S}}_{i,j}$. Similarly, $\text{ext}(h_{i\to j}-h_{i,j})$ extends $h_{i\to j}-h_{i,j}$ to scope ${\mathcal{C}}_j$ with $0$ entries on rows for ${\mathcal{C}}_j\setminus {\mathcal{S}}_{i,j}$.

If the phylogeny is a tree, performing these updates from the tips to the root corresponds to the recursive eqns (9), (10) and (11) of Mitov et al. [18], and to the propagation formulas (A.3)–(A.8) of Bastide et al. [59], who both considered the general linear Gaussian model (3.1).

At any point, a belief ${\displaystyle \text{C}(x;\mathit{K},h,g)}$ gives a local estimate of the conditional mean (${\displaystyle {\mathit{K}}^{-1}h}$) and conditional variance (${\displaystyle {\mathit{K}}^{-1}}$) of trait $X$ given data $\mathrm{Y}$, for ${\displaystyle \mathit{K}\succ 0}$. An exact belief, such that ${\displaystyle \text{C}(x;\mathit{K},h,g)\propto p_{\theta }(x\mid \mathrm{Y})}$, gives exact conditional estimates, that is: ${\displaystyle \mathrm{E}(X\mid \mathrm{Y})={\mathit{K}}^{-1}h}$ and ${\displaystyle \mathrm{v}\mathrm{a}\mathrm{r}(X\mid \mathrm{Y})={\mathit{K}}^{-1}}$.

5. Scalable approximate inference with loopy belief propagation

The previous examples focused on clique trees and the exact calculation of the likelihood. We now turn to the use of cluster graphs with cycles, or loopy cluster graphs, such as in figure 2c or 3c,d. BP on a loopy cluster graph, abbreviated as loopy BP, can approximate the likelihood and posterior distributions of ancestral values and can be more computationally efficient than BP on a clique tree.

Figure 3.

(a) Admixture graph $N$ from Lazaridis et al. [108, fig. 3] with $h=4$ reticulations (hybrid edges are coloured). $N$ has one non-trivial biconnected component (blob) $B$, induced by all its internal nodes except for the root. $B$ contains all four reticulations so $N$ has level $\ell =4$. (b–d) Various cluster graphs for the moralized blob $B^m$: (b) clique tree, (c) join-graph structuring with the maximum cluster size set to three, (d) LTRIP using the set of node families in $B$. Here sepsets (not shown) are the intersection of their incident clusters, and are small with one node only in (c) and (d). Purple boxes and edges, clusters and sepsets that contain node 8; red text, hybrid families.

(a). Calibration

Updating beliefs on a loopy cluster graph uses algorithm 1 in the same way as on a clique tree. A cluster graph is said to be calibrated when its normalized beliefs have converged (i.e. are unchanged by algorithm 1 along any edge). For calibration, neighbouring clusters ${\mathcal{C}}_i$ and ${\mathcal{C}}_j$ must have beliefs that are marginally consistent over the variables in their sepset ${\mathcal{S}}_{i,j}$:

$${\displaystyle \int {\beta }_{i}d({\mathcal{C}}_i\setminus {\mathcal{S}}_{i,j})={\tilde{\mu }}_{i\to j}\propto {\mu }_{i,j}\propto {\tilde{\mu }}_{j\to i}=\int {\beta }_{j}d({\mathcal{C}}_j\setminus {\mathcal{S}}_{i,j}).}$$

On a clique tree, calibration can be guaranteed at the end of a finite sequence of messages passed. Clique and sepset beliefs are then proportional to the posterior distribution over their variables, and can be integrated to compute the common normalization constant $\kappa ={\kappa }_i(=\int {\beta }_{i}d{\mathcal{C}}_i)={\kappa }_{j,k}(=\int {\mu }_{j,k}d{\mathcal{S}}_{j,k})$, which equals the likelihood. For loopy BP, calibration is not guaranteed. If it is attained, then we can similarly view cluster and sepset beliefs as unnormalized approximations of the posterior distribution over their variables, though the ${\kappa }_i$s and ${\kappa }_{j,k}$s may differ, grow unboundedly, and generally do not equal or estimate the likelihood. Gaussian models enjoy the remarkable property that, if calibration can be attained on a cluster graph, then the approximate posterior means (ancestral values) are guaranteed to be exact. By contrast, the posterior variances are generally inexact, and are typically underestimated [109–111], although we found them overestimated in our phylogenetic examples below (figure 6).

Successful calibration depends on various aspects, such as the features of the loops in the cluster graph, the factors in the model, and the scheduling of messages. For beliefs to converge, a proper message schedule requires that a message is passed along every sepset, in each direction, infinitely often (until stopping criteria are met) [111]. Multiple scheduling schemes have been devised to help reach calibration more often and more accurately. These can be data-independent (e.g. choosing a list of trees nested in the cluster graph that together cover all clusters and edges, then iteratively traversing each tree in both directions [110]) or adaptive (e.g. prioritizing messages between clusters that are further from calibration [112–115]).

(b). Likelihood approximation

To approximate the log-likelihood ${\displaystyle \mathrm{L}\mathrm{L}(\theta )=\log \int p_{\theta }(x)dx}$ from calibrated beliefs on cluster graph ${\mathcal{U}}^{*}=({\mathcal{V}}^{*},{\mathcal{E}}^{*})$, denoted together as $q=\{{\beta }_i,{\mu }_{i,j};{\mathcal{C}}_i\in {\mathcal{V}}^{*},\{{\mathcal{C}}_i,{\mathcal{C}}_j\}\in {\mathcal{E}}^{*}\}$, we can use the factored energy functional [26]:

$${\displaystyle {\displaystyle \tilde{F}(p_{\theta },q)=\sum _{{\mathcal{C}}_i\in {\mathcal{V}}^{\ast }}{\mathrm{E}}_{{\beta }_i}(\log {\psi }_i)+\sum _{{\mathcal{C}}_i\in {\mathcal{V}}^{\ast }}\mathrm{H}({\beta }_i)-\sum _{\{{\mathcal{C}}_i,{\mathcal{C}}_j\}\in {\mathcal{E}}^{\ast }}\mathrm{H}({\mu }_{i,j}).}}$$ (5.1)

Recall that ${\psi }_i$ is the product of factors ${\varphi }_v$ assigned to cluster ${\mathcal{C}}_i$. Here ${\displaystyle {\mathrm{E}}_{{\beta }_i}}$ denotes the expectation with respect to ${\beta }_i$ normalized to a probability distribution. $\mathrm{H}({\beta }_i)$ and $\mathrm{H}({\mu }_{i,j})$ denote the entropy of the distributions defined by normalizing ${\beta }_i$ and ${\mu }_{i,j}$, respectively. $\stackrel{\mathrm{~}}{F}(p_{\theta },q)$ has the advantage of involving local integrals that can be calculated easily: each over the scope of a single cluster or sepset. The justification for $\stackrel{\mathrm{~}}{F}(p_{\theta },q)$ comes from two approximations. First, following the expectation-maximization (EM) decomposition, ${\displaystyle \mathrm{L}\mathrm{L}(\theta )}$ can be approximated by the evidence lower bound (ELBO) used for variational inference [116]. For any distribution $q$ over the full set of variables, which are here the unobserved (latent) variables after absorbing evidence from the data, we have

$${\displaystyle \mathrm{L}\mathrm{L}(\theta )\ge \mathrm{E}\mathrm{L}\mathrm{B}\mathrm{O}(p_{\theta },q)={\mathrm{E}}_q(\log p_{\theta })+H(q).}$$

The gap ${\displaystyle \mathrm{L}\mathrm{L}(\theta )-\mathrm{E}\mathrm{L}\mathrm{B}\mathrm{O}(p_{\theta },q)}$ is the Kullback–Leibler divergence between $q$, and $p_{\theta }$ normalized to the distribution of the unobserved variables conditional on the observed data. The first approximation comes from minimizing this gap over a class of distributions $q$ that does not necessarily include the true conditional distribution. The second approximation comes from pretending that for a given distribution $q$ with a belief factorization

$${\displaystyle {\displaystyle q\propto \frac{\prod _{{\mathcal{C}}_i\in {\mathcal{V}}^{\ast }}{\beta }_i}{\prod _{\{{\mathcal{C}}_i,{\mathcal{C}}_j\}\in {\mathcal{E}}^{\ast }}{\mu }_{i,j}},}}$$

its marginal over a given cluster (or sepset) is equal to the normalized belief of that cluster (or sepset), simplifying ${\displaystyle {\mathrm{E}}_q(\log {\psi }_i)}$ to ${\displaystyle {\mathrm{E}}_{{\beta }_i}(\log {\psi }_i)}$ and simplifying ${\displaystyle {\mathrm{E}}_q(-\log {\beta }_i)}$ to $\mathrm{H}({\beta }_i)$. This simplification leads to the more tractable $\stackrel{\mathrm{~}}{F}(p_{\theta },q)$, in which each integral is of lower dimension, within the scope of a single cluster or sepset.

(c). Scalability versus accuracy: choice of cluster graph complexity

(i). Scalability, treewidth and phylogenetic network complexity

At the cost of exactness, loopy cluster graphs can offer greater computational scalability than clique trees because they allow for smaller cluster sizes, which reduces the complexity associated with belief updates. For example, consider a Gaussian model for $p$ traits: $\dim (x_v)=p$ at all nodes $v$ in the network. For a clique tree $\mathcal{U}$ with $m$ cliques and maximum clique size $k$, passing a message between neighbour cliques has complexity $\mathcal{O}(p^3{k}^3)$ and calibrating $\mathcal{U}$ has complexity $\mathcal{O}(m{p}^3{k}^3)$. Now consider a cluster graph ${\mathcal{U}}^{*}$ with $m^{*}$ clusters, $\mathcal{O}(m^{*})$ edges and maximum cluster size $k^{*}<k$. Then passing a message between neighbour cliques of ${\mathcal{U}}^{*}$ has complexity $\mathcal{O}(p^3{k^{*}}^3)$ so it is faster than on $\mathcal{U}$. But calibrating ${\mathcal{U}}^{*}$ now requires more belief updates because each edge needs to be traversed more than twice. If each edge is traversed in both directions $b$ times to reach convergence, then calibrating ${\mathcal{U}}^{*}$ has complexity $\mathcal{O}(b{m}^{*}{p}^3{k^{*}}^3)$. So if ${\mathcal{U}}^{*}$ has smaller clusters than $\mathcal{U}$ and if $(k/k^{*}{)}^3\gg b{m}^{*}/m$, then loopy BP on ${\mathcal{U}}^{*}$ runs faster than BP on $\mathcal{U}$. Loopy BP could be particularly advantageous for complex networks whose clique trees have large clusters.

Cluster graph construction determines the balance between scalability and approximation quality. At one end of the spectrum, the most scalable and least accurate are the factor graphs, also known as Bethe cluster graphs [117]. A factor graph has one cluster per factor ${\varphi }_v$ and one cluster per variable, and so has the smallest possible maximum clique size $k^{*}$ and each sepset reduced to a single variable. Various algorithms have been proposed for constructing cluster graphs along the spectrum (e.g. LTRIP [118]) (figure 3). Notably, join-graph structuring [119] spans the whole spectrum because it is controlled by a user-defined maximum cluster size $k^{*}$, which can be varied from its smallest possible value to a value large enough to obtain a clique tree.

At the other end of the spectrum, the best maximum clique size $k$ is ${\displaystyle 1+\mathrm{t}\mathrm{w}(G^m)}$, where ${\displaystyle \mathrm{t}\mathrm{w}(G^m)}$ is the treewidth of the moralized graph. Loopy BP becomes interesting when ${\displaystyle \mathrm{t}\mathrm{w}(G^m)}$ is large, making exact BP costly. Unfortunately, determining the treewidth of a general graph is NP-hard [120,121]. Heuristics such as greedy minimum-fill or nested dissection [122,123] can be used to obtain clique trees whose maximum clique size $k$ is near the optimum ${\displaystyle 1+\mathrm{t}\mathrm{w}(G^m)}$.

Different cluster graph algorithms could potentially be applied to the different biconnected components, or blobs [124] (e.g. LTRIP for one blob, clique tree for another), perhaps based on a blob’s attributes that are easy to compute. To choose between loopy versus exact BP, or between different cluster graph constructions more generally, one could use traditional complexity measures of phylogenetic networks as potential predictors of cost-effectiveness. For example, the reticulation number $h$ is straightforward to compute. In a binary network, where all internal non-root nodes have degree 3, $h$ is simply the number of hybrid nodes. More generally $h=|\{\text{hybrid edges}\}|-|\{\text{hybrid nodes}\}|$ [125]. The level of a network is the maximum reticulation number within a blob [126]. The network’s level ought to predict treewidth better than $h$ because a graph’s treewidth equals the maximum treewidth of its blobs [127], and moralizing the network does not affect its nodes’ blob membership. These phylogenetic complexity measures do not predict treewidth perfectly [128] except in simple cases, as shown below and proved in electronic supplementary material, §B.

Proposition 5.1. Let $N$ be a binary phylogenetic network with $h$ hybrid nodes, level $\ell$ and let $t$ be the treewidth of the moralized network $N^m$ obtained from $N$. For simplicity, assume that $N$ has no parallel edges and no degree-2 nodes other than the root.

(A0) If $\ell =0$ then $h=0$ and $t=1$.

(A1) If $\ell =1$ then $h\ge 1$ and $t=2$.

(A2) Let $v_1$ be a hybrid node with non-adjacent parents $u_1,u_2$. If $v_1$ has a descendant hybrid node $v_2$ such that one of its parents is not a descendant of either $u_1$ or $u_2$, then $\ell \ge 2$ and $t\ge 3$.

Level-1 networks have received much attention in phylogenetics because they are identifiable under various models under some mild restrictions [9,129–131]. Several inference methods limit the search to level-1 networks [9,132–134]. Since moralized level-1 networks have treewidth 2, exact BP is guaranteed to be efficient on them.

Beyond level-1, a network has a hybrid ladder (also called stack [135]) if a hybrid node $v_1$ has a hybrid child node $v_2$. By proposition 5.1, a hybrid ladder has the potential to increase the treewidth of the moralized network and decrease BP scalability, if the remaining conditions in (A2) are met. Related results in Chaplick et al. [136] are for undirected graphs that do not require prior moralization, and contain ladders defined as regular $2\times L$ grids. Their Observation 1, that a graph containing a non-disconnecting grid ladder of length $L\ge 2$ has treewidth at least 3, relies on a similar argument as for (A2). However, structures leading to the conditions in (A2) are more general, even before moralization. It may be interesting to extend some of the results from Chaplick et al. [136] to moralized hybrid ladders in rooted networks.

In figure 4 (right), $N_2$ has a hybrid ladder that does not meet all conditions of (A2) and has $t=2$. Outerplanar networks have a treewidth of at most $2$ [127], and if bicombining (hybrid nodes have exactly two parents), remain outerplanar after moralization. Networks in which no hybrid node is the descendant of another hybrid node in the same blob are called galled networks [137]. They provide more tractability to solve the cluster containment problem [138]. Here, galled networks would then never meet the assumptions of (A2) and it would be interesting to study their treewidth after moralization.

Figure 4.

Two binary networks with a hybrid ladder and $h=\ell =2$. $N_1$ satisfies (A2) of proposition 5.1 and $N_1^m$ has treewidth $t=3$. $N_2$ does not meet (A2) (see red/purple annotations) and $N_2^m$ has treewidth $t=2$. Stacking more hybrid ladders in the same way above $a$ and $b$ increases $h$ and $\ell$ but leaves $N_2^m$ outerplanar, keeping $t=2$.

We performed an empirical investigation of how $h$ and $\ell$ can predict the treewidth $t$ of the moralized network. Figure 5 shows that $t$ correlates with $h$ and $\ell$ on networks estimated from real data using various inference methods and on networks simulated under the biologically realistic birth–death-hybridization model [149,150], especially for complex networks. For networks with hundreds of tips ([151] lists several studies of this size), large maximum clique sizes $k\ge 30$ are not uncommon. By contrast, a Bethe cluster graph would have maximum cluster size $k^{*}=3$, so that $(k/k^{*}{)}^3\ge {10}^3$ would provide a large computational gain for loopy BP to be considered.

Figure 5.

We observe a positive sublinear relationship between a maximum clique size upperbound (from the greedy minfill heuristic) and the number of hybrids $h$ (a) or network level $\ell$ (b) on a combined sample of 11 empirical networks and 2509 simulated birth–death-hybridization networks. The empirical networks were sampled from Maier et al. [11, figs. 3(a–c) (left), 4(a-c) (left)] (reported as estimated by [139–144]), Lazaridis et al. [108, fig. 3], Nielsen et al. [12, fig. 3 (left)], Sun et al. [145, fig. 4(c)], Müller et al. [146, fig. 1(a)], Neureiter et al. [10, fig. 5(a)]; fit by these authors using ADMIXTOOLS [11,78], admixturegraph [147], OrientAGraph [148], contacTrees [10], Recombination [12], AdmixtureBayes [12]. The simulated networks were obtained by subsampling 10 networks per parameter scenario simulated by Justison & Heath [149], then filtering out networks of treewidth 1 (trees, possibly with parallel hybrid edges). The graphs are similar at high values because most networks have most of their hybrids contained in one large blob, leading to $h\approx \ell$. For example, $|\ell -h|\le 2$ in 99% of the networks with $h>10$.

(ii). Approximation quality with loopy belief propagation

We simulated data on a complex graph (40 tips, 361 hybrids) [146, fig. 1(a)] and a simpler graph (12 tips, 12 hybrids) [139, extended data fig. 4], then compared estimates from exact and loopy BP. For both networks, edges of length 0 were assigned the minimum non-zero edge length after suppressing any non-root degree-2 nodes. Trait values ${\displaystyle \mathbf{x}=({\mathrm{x}}_1,\dots ,{\mathrm{x}}_n)}$ at the tips were simulated from a BM with rate ${\sigma }^2=1$ and ${\mathrm{x}}_{\rho }=0$ at the root. Figure 6 shows the exact and approximate log-likelihood and conditional mean and variance of $x_{\rho }$ assuming a BM with rate ${\sigma }^2=1$ but improper prior $x_{\rho }\sim \mathcal{N}(0,\infty )$, using a greedy minimum-fill clique tree $\mathcal{U}$ and a cluster graph ${\mathcal{U}}^{*}$. Using a factor graph, calibration failed for the complex network (electronic supplementary material, §C, figure S3), so we used join-graph structuring to build ${\mathcal{U}}^{*}$. $\mathcal{U}$ can be calibrated in one iteration and the calculated quantities are exact (horizontal lines). By contrast, ${\mathcal{U}}^{*}$ requires multiple passes and gives approximations. Calibration required more iterations on the complex network ($h=361$) than on the simpler network (${\displaystyle h=12}$), as expected. But for both networks, the factored energy (5.1) approximated the log-likelihood very well. The distribution of the root state $x_{\rho }$ conditional on the data seems more difficult to approximate. The conditional mean was correctly estimated but required more iterations than the log-likelihood approximation on the complex network. The conditional variance was severely overestimated on the complex network and very slightly overestimated on the simpler network. As desired, the average computing time per belief update was lower on ${\mathcal{U}}^{*}$, although modestly so due to the clique tree $\mathcal{U}$ having many small clusters of size similar to those in ${\mathcal{U}}^{*}$ (electronic supplementary material, figure S4).

Figure 6.

Accuracy of loopy BP. Approximation of the conditional distribution of the root state $X_{\rho }$ (left and centre) and log-likelihood (right) using a greedy minimum-fill clique tree $\mathcal{U}$ and a join-graph structuring cluster graph ${\mathcal{U}}^{*}$ for two networks of varying complexity [139,146] as measured by their number of tips ($n$), level ($\ell$), number of hybrids ($h$), maximum clique size ($k$) and maximum cluster size ($k^{*}$). For $\mathcal{U}$, estimates are exact after one iteration and shown as horizontal red lines. For ${\mathcal{U}}^{*}$, estimates are shown over 20 (first row), 50 or 200 (second row) iterations. Each iteration consists of two passes through each spanning tree in a minimal set that jointly covers ${\mathcal{U}}^{*}$. In each plot, the two curves correspond to two different regularizations of initial beliefs (electronic supplementary material, §E, dotted: algorithm R1, solid: algorithm R2).

6. Leveraging belief propagation for efficient parameter inference

(a). Belief propagation for fast likelihood computation

In some particularly simple models, such as the BM on a tree, fast algorithms such as IC [34] or phylolm [16] can directly calculate the best-fitting parameters that maximize the restricted likelihood (REML), in a single tree traversal avoiding numerical optimization. For more general models, such closed-form estimates are not available. One product of BP is the likelihood of any fixed set of model parameters. BP can hence be simply used as a fast algorithm for likelihood computation, which can then be exploited by any statistical estimation technique, in a Bayesian or frequentist framework. Most of the tools cited in §2c use either direct numerical optimization of the likelihood [41,53,152] or sampling techniques such as Markov Chain Monte Carlo (MCMC) [1,14] for parameter inference.

BP also outputs the trait distribution at internal, unobserved nodes conditioned on the observed data at the tips. In addition to providing a tool for efficient ancestral state reconstruction, these conditional means and variances can be used for parameter inference, with approaches based on latent variable models such as EM [57], or Gibbs sampling schemes [3]. Although not currently used in the field of evolutionary biology to our knowledge, approaches based on approximate EM algorithms [153] and relying on loopy BP could also be used.

The linear Gaussian framework can also be useful for traits that rely on latent Gaussian liabilities, such as the threshold model for discrete traits (example 4.4), factor analysis [62], and phylogenetic structural equation models [151]. In a Bayesian context, the latent liabilities can be sampled at the tips, and then the conditional likelihood can be computed efficiently with Gaussian BP. A similar approach was successful on trees [3–5]. The BP framework can generalize such methods to phylogenetic networks.

(b). Belief propagation for fast gradient computation

As we show below, the conditional means and variances at ancestral nodes can be used to efficiently compute the gradient of the likelihood [154]. The gradient of the likelihood can help speed up inference in many different statistical frameworks [155]. In a phylogenetic context, gradients have been used to improve maximum likelihood estimation [60,61], Bayesian estimation through Hamiltonian Monte Carlo (HMC) approaches [4,58,59], or variational Bayes approximations [156]. Although automatic differentiation can be used on trees for some models [157], direct computations of the gradient using BP-like algorithms have been shown to be more efficient in some contexts [158]. After recalling Fisher’s identity to calculate gradients after BP calibration, we illustrate its use on the BM model (univariate or multivariate) where it allows for the derivation of a new analytical formula for the REML parameter estimates.

(i). Gradient computation with Fisher’s identity

In a phylogenetic context, latent variables are usually internal nodes, while observed variables are leaves. We write ${\displaystyle Y=\{X_{v,j}:\text{trait}j\text{observed at}v\in V\}}$ the set of observed variables. Fisher’s identity provides a way to link the gradient of the log-likelihood of the data ${\displaystyle \mathrm{L}\mathrm{L}(\theta )=\log p_{\theta }(Y)}$ at parameter $\theta$, with the distribution of all the variables conditional on the observations $Y$. We refer to Cappé et al. [159, ch. 10] or Barber [155, ch. 11] for general introductions on Markov models with latent variables. Under broad assumptions, Fisher’s identity states (see proposition 10.1.6 in [159], or §11.6 in [155]):

$${\displaystyle {{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(Y)\right]|}_{{\theta }^{\prime }=\theta }={\mathrm{E}}_{\theta }\left[{{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(X_v;v\in V)\right]|}_{{\theta }^{\prime }=\theta }|Y\right],}$$

where ${{\nabla }_{{\theta }^{\prime }}[f({\theta }^{\prime })]|}_{{\theta }^{\prime }=\theta }$ denotes the gradient of $f$ with respect to the generic parameters ${\theta }^{\prime }$ and evaluated at ${\theta }^{\prime }=\theta$, and ${\displaystyle {\mathrm{E}}_{\theta }[\bullet |Y]}$ the expectation conditional on the observed data under the model parametrized by $\theta$, which is precisely where the output from BP can be used. Plugging in the factor decomposition from the graphical model (4.1), we get

$${\displaystyle {\displaystyle {{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(Y)\right]|}_{{\theta }^{\prime }=\theta }=\sum _{v\in V}{\mathrm{E}}_{\theta }\left[{{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log {\varphi }_v(X_v|X_u,{\theta }^{\prime };u\in \mathrm{p}\mathrm{a}(v))\right]|}_{{\theta }^{\prime }=\theta }|Y\right].}}$$ (6.1)

While (6.1) applies to the full vector of all model parameters, it can also be applied to take the gradient with respect to a single parameter $\theta$ of interest, keeping the other parameters fixed. For instance, we can focus on one rate matrix ${\displaystyle \mathbf{\Sigma }}$ of a BM model, or one primary optimum of an OU model. Special care needs to be taken for gradients with respect to structured matrices, such as variance matrices that need to be symmetric (e.g. [59]) or with a sparse inverse under structural equation modelling for high-dimensional traits [160].

For models where the conditional expectation of the factor in (6.1) has a simple form, this formula is the key to an efficient gradient computation. In particular, for discrete traits as in example 4.2, the expectation becomes a sum of a manageable number of terms, local to a cluster, weighted by the normalized cluster belief after calibration [26, ch. 19].

(ii). Gradient computation for linear Gaussian models

For linear Gaussian models (3.1), log-factors can be written as quadratic forms (4.3), so their derivatives have analytical formulas (see electronic supplementary material, §D). The conditional expectation in (6.1) then only depends on the joint first and second-order moments of the variables ${\displaystyle (X_v,X_{\mathrm{p}\mathrm{a}(v)})}$ in a cluster, which are known as soon as the beliefs are calibrated. When the graph is a tree, Bastide et al. [59] exploited this formula to derive gradients in the general linear Gaussian case. However, they did not use the complete factor decomposition (4.1), but instead an ad hoc decomposition that only works when the graph is a (binary) tree, and exploits the split partitions defined by the tree. By contrast, the present approach gives a recipe for the efficient gradient computation of any linear Gaussian model on any network, as soon as beliefs are calibrated.

In the special case where the process is a homogeneous BM (univariate or multivariate) on a network with a weighted-average merging rule (3.2), a constant rate ${\displaystyle \mathbf{\Sigma }}$, no missing data at the tips and, if present, within-species variation that is proportional to ${\displaystyle \mathbf{\Sigma }}$, then the gradient with respect to ${\displaystyle \mathbf{\Sigma }}$ takes a particularly simple form. Setting this gradient to zero, we find an analytical formula for the REML estimate of ${\displaystyle \mathbf{\Sigma }}$ and for the ML estimate of the ancestral mean ${\mu }_{\rho }$ (electronic supplementary material, §D.3). In a phylogenetic regression setting, a similar formula can be found for the ML estimate of coefficients (electronic supplementary material, §D.4). Efficient algorithms such as IC and phylolm already exist to compute these quantities on a tree in a single traversal. Here, our formulas need two traversals but remain linear in the number of tips, and because they rely on a general BP formulation, they apply to networks with reticulations. Fisher’s identity and BP hence offer a general method for gradient computation and could lead to analytical formulas for other simple models. Such efficient formulas could alleviate numerical instabilities observed in software such as mvSLOUCH, which experienced a significant failure rate for the BM on trees with a large number of traits [161].

(iii). Hessian computation with Louis’s identity

Second-order derivatives can improve both maximum likelihood and Bayesian statistical inference methods. Efficient Hessian computation methods have been developed for branch lengths in sequence evolution models [60,61], and recently for continuous trait model parameters on trees [162]. Using BP, the Hessian of the log-likelihood with respect to the parameters can also be obtained as a conditional expectation of the Hessian of the complete log-likelihood:

$${\displaystyle \begin{array}{l}{\left\{{\mathrm{\nabla }}_{{\theta }^{\prime }}^2\left[\log p_{{\theta }^{\prime }}(Y)\right]+{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(Y)\right]{\left[{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(Y)\right]\right]}^{\mathrm{\top }}\right\}|}_{{\theta }^{\prime }=\theta }=\\ {\mathrm{E}}_{\theta }\left[\{{\mathrm{\nabla }}_{{\theta }^{\prime }}^2\left[\log p_{{\theta }^{\prime }}(X_v;v\in V)\right]+{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(X_v;v\in V)\right]{\left[{\mathrm{\nabla }}_{{\theta }^{\prime }}\left[\log p_{{\theta }^{\prime }}(X_v;v\in V)\right]\right]}^{\mathrm{\top }}\}{|}_{{\theta }^{\prime }=\theta }|Y\right].\end{array}}$$

This so-called Louis identity [159] also simplifies under the factor decomposition (4.1), and leads to tractable formulas in simple Gaussian or discrete cases.

(c). Belief propagation for direct Bayesian parameter inference

Likelihood or gradient-based approaches require careful analytical computations to get exact formulas in any new model within the class of linear Gaussian graphical models, depending on the parameters of interest [59]. One way to alleviate this problem is to use a Bayesian framework, and expand the graphical model to include both the phylogenetic network and the evolutionary parameters, which are seen as random variables themselves (e.g. as in [92]). Then, inferring parameters amounts to learning their conditional distribution in this larger graphical model. In this setting, the output of interest from BP is not the likelihood but the distribution of random variables (evolutionary parameters primarily) conditional on the observed data.

Exact computation may not be possible in this extended graphical model, because it is typically not linear Gaussian and the graph’s treewidth can be much larger than that of the phylogenetic network when one parameter (e.g. the evolutionary rate) affects multiple node families. Therefore, approximations may need to be used. For example, ‘black box’ optimization techniques rely on variational approaches to reach a tractable approximation of the posterior distribution of model parameters [116]. The conditional distribution of unobserved variables, provided by BP, facilitates the noisy approximation of the variational gradient that can be used to speed up the optimization of the variational Bayes approximation.

7. Challenges and extensions

(a). Degeneracy

While our implementation provides a proof-of-concept, various technical challenges still need to be solved. Much of the literature on BP focuses on factor graphs, which failed to converge for one of our example phylogenetic networks. More work is needed to better understand the convergence and accuracy of alternative cluster graphs, and on other choices that can substantially affect loopy BP’s efficiency, such as scheduling. Below, we focus on implementation challenges due to degeneracies.

For the message ${\stackrel{\mathrm{~}}{\mu }}_{i\to j}$ to be well-defined in step 1 of Gaussian BP, the belief of the sending cluster must have a precision matrix ${\displaystyle \mathit{K}}$ in (4.3) with a full-rank submatrix with respect to the variables to be integrated out (${\displaystyle {\mathit{K}}_I}$ in algorithm 2). This condition can fail under realistic phylogenetic models, due to two different types of degeneracy.

The first type arises from deterministic factors: when ${\displaystyle {\mathit{V}}_v=0}$ in (3.1) and $X_v$ is determined by the states at parent nodes ${\displaystyle X_{\mathrm{p}\mathrm{a}(v)}}$ without noise, e.g. when all of $v$’s parent branches have length 0 in standard phylogenetic models. This is expected at hybridization events when both parents have sampled descendants in the phylogeny because the parents and hybrid need to be contemporary of one another. This situation is also common in admixture graphs [11] due to a lack of identifiability of hybrid edge lengths from $f$ statistics, leading to a ‘zipped-up’ estimated network in which the estimable composite length parameter is assigned to the hybrid’s child edge [131]. With this degeneracy, $X_v$ has infinite precision given its parents, that is, ${\displaystyle \mathit{K}}$ has some infinite values. The complications are technical, but not numerical. For example, one can use a generalized canonical form that includes a Dirac distribution to capture the deterministic equation of $X_v$ given ${\displaystyle X_{\mathrm{p}\mathrm{a}(v)}}$ from (3.1). Then BP operations need to be extended to these generalized canonical forms, as done in Schoeman et al. [163] (illustrated in electronic supplementary material, §F). One could also modify the network by contracting internal tree edges of length 0. At hybrid nodes, adding a small variance to ${\displaystyle {\mathit{V}}_v}$ would be an approximate yet biologically realistic approach.

The second type of degeneracy arises when the precision submatrix ${\displaystyle {\mathit{K}}_I}$ is finite but not of full rank. In phylogenetic models, this is frequent at initialization (4.2). For example, consider a cluster of three nodes: a hybrid $v$ and its two parents. By (4.4) we see that rank${\displaystyle ({\mathit{K}}_v)\le p}$. So at initialization with belief ${\varphi }_v$, ${\displaystyle {\mathit{K}}_I}$ is degenerate if we seek to integrate out $|I|=2$ nodes, which would occur if the cluster is adjacent to a sepset containing only one parent of $v$. This situation is typical of factor graphs. Initial beliefs would also be degenerate with ${\displaystyle \mathit{K}=\mathbf{0}}$ for any cluster that is not assigned any factor by (4.2). This may occur if there are more clusters than node families, or if the graph has nested redundant clusters (e.g. from join-graph structuring). In some cases, a schedule may avoid these degeneracies, guaranteeing a well-defined message at each BP update. On a clique tree, a schedule based on a post-order traversal has this guarantee, provided that all $p$ traits are observed at all leaves. But generally, it is unclear how to find such a schedule. Another approach is to simply skip a BP update if its message is ill-defined, though there is no guarantee that the sending cluster will eventually have a well-behaved belief to pass the message later. A robust option is to regularize cluster beliefs, right after initialization (4.2) or during BP, by increasing some diagonal elements of ${\displaystyle \mathit{K}}$ to make ${\displaystyle {\mathit{K}}_I}$ of full rank. To maintain the probability model, this cluster belief regularization is balanced by a similar modification to a corresponding sepset. Electronic supplementary material, §E, describes two such approaches that appear to work well in practice, although theoretical guarantees have not been established.

(b). Loopy belief propagation is promising for discrete traits

We focused on Gaussian models in this paper, for which the ‘lazy’ matrix approach is polynomial. For discrete trait models, the computational gains from loopy BP can be much greater because alternative approaches are not polynomial in general networks. For a trait with $c$ states ($c=2$ for a binary trait as in example 4.2), passing a message has complexity $\mathcal{O}(c^k)$ where $k$ is the sending cluster size. Thus, cluster graphs with small clusters can bring exponential computational gains. Even exact BP can bring significant computational gains to existing approaches that rely on other means to reduce complexity. For example, the model without ILS used in Allen-Savietta and Lutteropp et al. [65,66] is a mixture model, so the network likelihood can be calculated as a weighted average of tree likelihoods for which exact BP takes linear time. This approach scales exponentially with $h$ because there are typically $\mathcal{O}(2^h)$ trees displayed in a network. By contrast, the complexity of BP on a clique tree of maximum clique size $k$ is $\mathcal{O}(n{c}^k)$, thus parametrized by the treewidth $t$ of the moralized network instead of $h$ ($t=k-1$ for an optimal clique tree). Given our empirical evidence that $t$ grows more slowly than $h$ or the network’s level $\ell$ in biologically realistic networks (figure 5), exact BP could achieve significant computational gains and loopy BP substantially more.

A BP approach is already used in momi2 by Kamm et al. [72], who use a clique tree built from a node ordering by age from youngest to oldest, to get conditional likelihoods of the derived allele count under a Moran model (without mutation). The mutation-with-ILS model in SnappNet can also be reframed as a graphical model on a graph expanded from the phylogenetic network (as shown in example 4.3 and electronic supplementary material, §A). Accordingly, the BP-like algorithm in Rabier et al. [70] has complexity controlled by the network’s scanwidth, a parameter introduced by Berry et al. [164]. Using regular BP on more optimal clique trees and loopy BP on cluster graphs may help speed up computations even more.

Also related is the algorithm in Scornavacca & Weller [128], which uses a clique tree to solve a parsimony problem. In this non-probabilistic setting, it is unclear how cluster graphs could be leveraged to speed up algorithms as they do in loopy BP.

To deal with computational intractability, the most widely used probabilistic methods to infer networks from DNA sequences are based on composite likelihoods [8,9] or summary statistics like $f$ statistics [11,12], leading to a lack of identifiability for parts of the network topology and some of its parameters [9,129,131,165–167]. These identifiability issues should be alleviated if using the full data becomes tractable thanks to exact or loopy BP.

Ethics

This work did not require ethical approval from a human subject or animal welfare committee.

Data accessibility

Code to reproduce figures is is archived at [168] and available as a GitHub repository [169]. It uses a Julia package for Gaussian BP on phylogenetic networks, version 0.0.1, which is archived at [170] and available as a GitHub repository [171].

Supplementary material is available online [172].

Declaration of AI use

We have not used AI-assisted technologies in creating this article.

Authors’ contributions

B.T.: conceptualization, data curation, formal analysis, investigation, methodology, software, visualization, writing—original draft, writing—review and editing; P.B.: conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, software, visualization, writing—original draft, writing—review and editing; C.A.: conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, software, visualization, writing—original draft, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This work was supported in part by the National Science Foundation (DMS 2023239 to C.A.) and by the University of Wisconsin-Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation. C.A. visited P.B. at the University of Montpellier thanks to support from the I-SITE MUSE through the Key Initiative 'Data and Life Sciences'.