A Maximum Likelihood Method for Reconstruction of the Evolution of Eukaryotic Gene Structure

Abstract

Spliceosomal introns are one of the principal distinctive features of eukaryotes. Nevertheless, different large-scale studies disagree about even the most basic features of their evolution. In order to come up with a more reliable reconstruction of intron evolution, we developed a model that is far more comprehensive than previous ones. This model is rich in parameters, and estimating them accurately is infeasible by straightforward likelihood maximization. Thus, we have developed an expectation-maximization algorithm that allows for efficient maximization. Here, we outline the model and describe the expectation-maximization algorithm in detail. Since the method works with intron presence–absence maps, it is expected to be instrumental for the analysis of the evolution of other binary characters as well.

1. Introduction

In eukaryotes, many protein-coding genes have their coding sequence broken into pieces – the exons – separated by the non-coding spliceosomal introns. These introns are removed from the nascent pre-mRNA and the exons are spliced together to form the intronless mRNA by the spliceosome, a large and elaborate macromolecular complex comprising several small RNA molecules and numerous proteins. No spliceosomal introns have ever been found in prokaryotes, and there are no eukaryotes with a completely sequenced genomes, not even the very basal ones, which would not possess introns (1–3) and the accompanying splicing machinery (4).

Despite the introns being such a remarkable idiosyncrasy of eukaryotic genomes, their origin and evolution are not thoroughly understood (5, 6). It is generally accepted that introns can be regarded as units of evolution and that their presence/absence pattern is a result of stochastic processes of loss and gain. However, the nature of these processes is vigorously debated. Recent large-scale attempts to study these processes using extant eukaryotic genomes led to incongruent conclusions.

In a study on reconstruction of intron evolution, Rogozin et al. (7) analyzed ~700 sets of intron-bearing orthologous genes from eight eukaryotic species. The multiple alignment of the orthologs within each set was computed, and the intron positions were projected on the alignments to form presence/absence maps. Using Dollo parsimony to infer ancestral states, these authors observed a diverse repertoire of behaviors. Some lineages endured extensive losses, while others experienced mostly gain events. Early forbearers, such as the last common ancestor of multicellular life, were shown to be relatively intron-rich. This work suggested that both gain and loss of introns played significant roles in shaping the modern eukaryotic gene structure. However, as these inferences rely upon the Dollo parsimony reconstruction, the number of gains in terminal branches (leaves of the phylogenetic tree) is overestimated, resulting in underestimation (potentially, significant) of the number of introns in ancient lineages.

The same data set was analyzed by Roy and Gilbert (8, 9) using a different methodology. They adopted a simple evolutionary model, according to which different lineages are associated with different loss and gain probabilities. Using a variation on maximum likelihood estimation, they obtained considerably higher estimates for the number of introns in early eukaryotes and a correspondingly lower level of gains in all lineages, i.e., a clear dominance of loss events in the evolution of eukaryotic genes. Roy and Gilbert have substantially simplified the mathematics involved in the estimation procedure, at the expense of introducing into the computation considerations of parsimony, which yielded an inference technique that is a hybrid between parsimony and maximum likelihood. This hybrid, however, excludes from consideration different evolutionary scenarios, resulting in inflated estimates of the number of introns in early eukaryotes (10).

The model of Roy and Gilbert is branch-specific, i.e., it assumes that the gain and loss rates depend only on the branch, thus tacitly presuming that all genes behave identically with respect to intron gain and loss. Exactly the inverse approach was adopted by Qiu et al. (11). These authors developed a gene-specific model, whereby different gene families are characterized by different rates of intron gain and loss, but for a particular gene these rates are constant across the entire phylogenetic tree. They used a different data set combined with a Bayesian estimation technique and concluded that almost all extant introns were gained during the eukaryotic evolution. This suggests evolution is dominated by intron gain events with few losses. However, the validity of a gene-specific model is disputable as it is hard to reconcile with the accumulating evidence on large differences between lineages (12–15).

Recently, two maximum likelihood estimation techniques have been developed for essentially the same branch-specific evolutionary model as the one of Roy and Gilbert. Csuros (10) used a direct approach, while Nguyen et al. (16) developed an expectation-maximization algorithm. Both methods encountered the same problem of estimating the number of unobserved intronless sites. Each employed a technically different but conceptually similar method to evaluate this number. Both techniques were applied to the eight-species data of Rogozin et al. (7), yielding very close estimates. As expected, these methods predict intron occupancy level of ancient lineages higher than those predicted by Dollo parsimony and lower than those predicted by the hybrid technique of Roy and Gilbert. Notably, these estimates are generally closer to those obtained using Dollo parsimony, and they imply an evolutionary landscape comprising both losses and gains, with some excess of gains.

While the Dollo parsimony (7) and the hybrid technique of Roy and Gilbert (8, 9) showed some methodological biases, the other analyses of intron evolution (10, 11, 16) used well-established estimation techniques. Nevertheless, these studies kept yielding widely diverging inferences. The reason seems to be the differences in the underlying evolutionary models, neither being sufficient to describe the complex reality of intron evolution. The branch-specific model fails to account for important differences between genes, whereas the gene-specific model ignores the sharp differences between lineages. Additionally, rate variability between sites, known to be an important factor in other fields of molecular evolution (17, 18), should be taken into account also in the evolution of gene structure. This is particularly important for intron gain in light of the accumulating evidence in favor of the proto-splice model, according to which new introns are preferentially inserted inside certain sequence motifs (19–21). This means that sites could dramatically differ in their gain rate depending on their position relative to a proto-splice site.

Here we describe a model of evolution that takes into consideration all of the above factors. In order to efficiently estimate the model parameters by maximum likelihood, we have developed an expectation-maximization algorithm. We also compiled a data set that is considerably larger than previously used ones, consisting of 400 sets of orthologous genes from 19 eukaryotic species. Applying our algorithm to this data set, we obtained high-precision estimates, revealing a fascinating evolutionary history of gene structure, where both losses and gains played significant roles albeit the contribution of losses was somewhat greater. Moreover, we identified novel properties of intron evolution: (i) all eukaryotic lineages share a common, universal, mode of intron evolution, whereby the loss and gain processes are positively correlated. This suggests that the mechanisms of intron gain and loss share common mechanistic components. In some lineages, additional forces come into play, resulting either in elevated loss rate or in elevated gain rate. Lineages exhibiting an increased loss rate are dispersed throughout the entire phylogenetic tree. In contrast, lineages with excessive gains are much rarer, and all of them are ancient. (ii) Intron loss rates of individual genes show no correlation with any other genomic property. By contrast, intron gain rate of individual genes show several remarkable relationships, not always easily explained. In brief, intron gain rate is positively correlated with expression level, negatively correlated with sequence evolution rate, and negatively correlated with the gene length. Moreover, genes of apparent bacterial origin have significantly lower rates of intron gain than genes of archaeal origin. (iii) We showed that the remarkable conservation of intron positions is, mainly (~90%), due to shared ancestry, and only in a minority of the cases (~10%), due to parallel gain at the same location. (iv) We determined that the density of potential intron insertion sites is about 1 site per 7 nucleotides.

2. Materials

The algorithm learns the parameters of the model by comparing the structure of orthologous genes in extant species. To carry out this comparison, it requires two sets of input data, to be described in this section. The first is a phylogenetic tree, defining topological relationships between a set of eukaryotic species. The second is a collection of genes, for which one can identify orthologs in at least a subset of the species above.

2.1. Multiple Alignments

Suppose that we have G sets of aligned orthologous genes from S species. To represent the gene structure, we transform these alignments into intron presence–absence maps by substituting for each nucleotide (or amino acid) 0 or 1, depending on whether an intron is present or absent in the respective position. We allow for missing data by using a third symbol (*), and consequently a gene might be included in the input data even if it is missing in part of the species. Every site in an alignment, called pattern, is a vector of length S over the alphabet (0,1,*). Let Ω be the total number of unique patterns in the entire set of G alignments, denoted ω₁, . . . , ω_Ω, and let n_gp count the number of times pattern ω_p is found in the multiple alignment of gene g. Assuming that the sites evolve independently, the set M_g = (n_g1, . . . , n_gΩ) fully characterizes the multiple alignment of the gth gene. Thus, all the relevant information about the multiple alignments is captured by the list of unique patterns ω₁, . . . , ω_Ω, and the list of vectors M₁, . . . , M_G.

2.2. Phylogenetic Tree

Let T be a rooted bifurcating phylogenetic tree with S leaves (terminal nodes) corresponding to the S species above. The total number of nodes in T is N = 2S – 1, and we index them by t = 0, 1, . . . , N – 1, with the convention that zero is the root node. The state of node t is described by the random variable q_t, which can take the values 0 and 1 (and * in leaves). We use V_t for the set of all leaves such that node t is among their ancestors. The entire collection of leaves is, obviously, V₀. The parent node of t is denoted P(t). We use the special notations $q_t^P$ and $V_t^P$ for qp_(t) and V_P(t), respectively. Analogously, the two direct descendents of node t are denoted L(i) and R(i), and we use the special notations $q_t^L,q_t^R,V_q^L$, and $V_q^R$ for q_L(t), q_R(t), V_L(t), and V_R(t), respectively. We index branches by the node into which they are leading, and use Δ_t to denote the length (in time units) of the tth branch. We assume that the tree topology, as well as all the branch lengths Δ₁, . . . , Δ_N–1 are known.

3. Methods

3.1. The Probabilistic Model

A graphical model is a mathematical graph whose nodes symbolize random variables, and whose branches describe dependence relationships between them (22). A bifurcating phylogenetic tree, when viewed as a graphical model, depicts the probabilistic model

$$\mathrm{Pr}\left(q_0\right)\prod _{t=1}^{N-1}\mathrm{Pr}(q_t\mid q_t^P).$$ [1]

We use the notation π_i = Pr(q₀ = i) to describe the prior probability of the root, and $A_{ij}(g,t)=\mathrm{Pr}(q_t=j\mid q_t^P=i,g)$ to describe the transition probability for gene g along branch t. In our model, we assume that the transition probability depends on both the gene and the branch, and that it takes the explicit form

$$A(g,t)=\left(\begin{array}{cc}\hfill 1-{\xi }_t(1-e^{-{\eta }_g{\Delta }_t})\hfill & \hfill {\xi }_t(1-e^{-{\eta }_g{\Delta }_t})\hfill \\ \hfill 1-(1-{\varphi }_t)e^{-{\theta }_g{\Delta }_t}\hfill & \hfill (1-{\varphi }_t)e^{-{\theta }_g{\Delta }_t}\hfill \end{array}\right).$$ [2]

Here, η_g and θ_g are nonnegative parameters, determining the intron gain and loss rates, respectively, of gene g. Complementarily, ξ_t and φ_t determine the intron gain and loss coefficients of branch t, respectively, and are bound to the range 0 ≤ ξ_t, φ_t ≤ 1. The probability of an intron present in gene g at the beginning of branch t to be retained along the branch is $(1-{\varphi }_t)e^{-{\theta }_g{\Delta }_t}$, that is, it is retained only if the branch does not lose it (with probability 1 – φ_t), and also the gene does not lose it (with probability $e^{-{\theta }_g{\Delta }_t}$). This comes to reflect a reality where strong forces to strip a gene off its introns will be practically unaffected by the particular lineage, and, oppositely, strong forces to strip a lineage off its introns will be practically unaffected by the particular gene. In the same spirit, the probability of an intron to be gained in gene g along branch t is ${\xi }_t(1-e^{-{\eta }_g{\Delta }_t})$, that is, it is gained only if both the branch “approves” it (with probability ξ_t) and the gene “approves” it (with probability $1-e^{-{\eta }_g{\Delta }_t}$).

In other fields of molecular evolution, it was long realized that analysis precision improves if one allows for rate variability across sites (17, 18). Typically, such rate variability is modeled by introducing a rate variable, r, which scales, for each site, the time units of the phylogenetic tree, Δ_t ← r · Δ_t. This rate variable is a random variable, distributed according to a distribution function with nonnegative domain and unit mean, typically the unit-mean gamma distribution. The rate variability reflects the idea that sites differ in their rate of evolution. Specifically, there are fast-evolving sites (r > > 1), as well as slow-evolving ones (r < < 1). In our model of intron evolution we extend this idea by assuming that the gain and loss processes are subject to rate variability, independently of each other. Hence, a site can have any combination of gain and loss rates. To accommodate this idea, we use two independent rate variables, r^η and r^θ, that are used to scale, for each site, the gene-specific gain rate, ${\eta }_g\leftarrow r^{\eta }\cdot {\eta }_g$, and the gene-specific loss rate, ${\theta }_g\leftarrow r^{\theta }\cdot {\theta }_g$. We further assume that the distributions of these rate variables are independent of the genes, and are explicitly given by

$$\begin{array}{cc}\hfill r^{\eta }& \sim \nu \delta \left(\eta \right)+(1-\nu )\Gamma (\eta ;{\lambda }_{\eta })\hfill \\ \hfill r^{\theta }& \sim \Gamma (\theta ;{\lambda }_{\theta }).\hfill \end{array}$$ [3]

Here, Γ(x, λ) is the unit-mean gamma distribution of variable x with shape parameter λ, δ(x) is the Dirac delta-function, and η is the fraction of sites that are assumed to have zero gain rate. These latter sites, denoted invariant sites, reflect these sites that are not a proto-splice site (19–21). Intron loss does not have an invariant counterpart, as the assumption is that once an intron is gained, it can always be lost. Therefore, the loss rate variable is assumed to be distributed according to a gamma distribution, which is by far the most popular in describing rate variability (17, 18, 23).

In practice, the rate distributions in Eq. [3] are rendered discrete (24). We assume that the gain rate variable can take K_η discrete values $r_1^{\eta }=0,r_2^{\eta },\dots ,r_{K_{\eta }}^{\eta }$ with probabilities $f_1^{\eta }=\nu ,f_2^{\eta }\dots ,f_{K_{\eta }}^{\eta }$ such that ${\sum }_{k=1}^{K_{\eta }}{f}_k^{\eta }=1$. Analogously, we assume that the loss rate variable can take K_θ discrete values $r_1^{\theta },\dots ,r_{K_{\theta }}^{\theta }$ with probabilities $f_1^{\theta },\dots ,f_{K_{\theta }}^{\theta }$ such that ${\sum }_{k=1}^{K_{\theta }}{f}_k^{\theta }=1$. For a particular gain rate value $r_k^{\eta }$, we denote the actual gain rate $r_k^{\eta }\cdot {\eta }_g$ by ${\eta }_{kg}$. Similarly, for a particular loss rate value $r_k^{\theta }$, we denote the actual loss rate $r_k^{\theta }\cdot {\theta }_g$ by θ_kg.

For notational clarity, we aggregate the model parameters into a small number of sets. To this end, let Ξ_t = {ξ_t, φ_t} be the set of parameters that are specific for branch t, and let Ξ = (Ξ₁, . . . , Ξ_{N – 1}) be the set of all branch-specific parameters. Similarly, let ${\Psi }_g=({\eta }_g,{\theta }_g)$ be the set of parameters that are specific for gene g, and let Ψ = (Ψ₁ . . . ,Ψ_G) be the set of all gene-specific parameters. Additionally, we denote by Λ = (ν, λ_η, λ_θ) the parameters that determine the rate variability. When the distinction between the different sets of parameters is irrelevant, we shall use Θ = (Ξ, Ψ, Λ) as the set of all the model's parameters. We achieve further succinctness in notations by denoting the actual gene-specific rate values for particular values $r_k^{\eta }$ and $r_{k^{\prime }}^{\theta }$ of the rate variables as ${\Psi }_{k{k}^{\prime }g}=({\eta }_{kg},{\theta }_{k^{\prime }g})$.

3.2. The EM Algorithm

For each site, the S leaves form a set of observed random variables, their states being described by the corresponding pattern ω_p. The state of all the internal nodes, denoted σ, form a set of hidden random variables, that is, random variables whose state is not observed. In order to account for rate variability across sites, we associate with each pattern two hidden random variables, ${\rho }_p^{\eta }$ and ${\rho }_p^{\eta }$, that determine the value of the rate variables in that site. To sum up, the observed random variables are ω_p, and the hidden random variables are $(\sigma ,{\rho }_p^{\eta },{\rho }_p^{\theta })$.

We assume that sites within a gene, as well as the genes themselves, evolve independently. Therefore, the total likelihood can be decomposed as

$$L(M_1,\dots ,M_G\mid ϴ)=\prod _{g=1}^{G}L(M_g\mid \Xi ,{\Psi }_g,\Lambda )=\prod _{g=1}^G\prod _{p=1}^{\Omega }L{({\omega }_p\mid \Xi ,{\Psi }_g,\Lambda )}^{n_{gp}}.$$

and so

$$\text{log}L(M_1,\dots ,M_G\mid ϴ)=\sum _{g=1}^G\sum _{p=1}^{\Omega }{n}_{gp}\text{log}L({\omega }_p\mid \Xi ,{\Psi }_g,\Lambda ).$$ [4]

According to the well-known EM paradigm (25) log L(M₁, . . . , M_G|Θ) is guaranteed to increase as long as we maximize the auxiliary function

$$\mathcal{Q}(ϴ,{ϴ}^0)=\sum _{g=1}^G\sum _{p=1}^{\Omega }{n}_{gp}{\mathcal{Q}}_{gp}(\Xi ,{\Psi }_g,\Lambda ,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0),$$ [5]

where

$${\mathcal{Q}}_{gp}(\Xi ,{\Psi }_g,\Lambda ,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0)=\sum _{\sigma ,{\rho }_p^{\eta },{\rho }_p^{\theta }}\mathrm{Pr}(\sigma ,{\rho }_p^{\eta },{\rho }_p^{\theta }\mid {\omega }_p,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0)\text{log}\mathrm{Pr}({\omega }_p,\sigma ,{\rho }_p^{\eta },{\rho }_p^{\theta }\mid \Xi ,{\Psi }_g,\Lambda ).$$ [6]

Using some manipulations (see Note 1), this can be written as

$${\mathcal{Q}}_{gp}\left(\Xi ,{\Psi }_g,\Lambda ,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0\right)=\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}\left[\mathrm{Pr}\left({\rho }_p^{\eta }=k,{\rho }_p^{\theta }=k^{\prime }\mid {\omega }_p,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0\right)\right].\left[\sum _{\sigma }\mathrm{Pr}\left(\sigma \mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0\right)\cdot \{\text{log}{f}_k^{\eta }+\text{log}{f}_{k^{\prime }}^{\theta }+\text{log}\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})\}\right].$$

Denoting by $w_{gpk{k}^{\prime }}$ and ${\mathcal{Q}}_{gpk{k}^{\prime }}$ the first and second square brackets, respectively, this expression becomes

$${\mathcal{Q}}_{gp}(\Xi ,{\Psi }_g,\Lambda ,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0)=\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}{w}_{gpk{k}^{\prime }}{\mathcal{Q}}_{gpk{k}^{\prime }},$$ [7]

and consequently

$$\mathcal{Q}(ϴ,{ϴ}^0)=\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}{n}_{gp}{w}_{gpk{k}^{\prime }}{\mathcal{Q}}_{gpk{k}^{\prime }}.$$ [8]

3.2.1. The E-Step

In this step we compute the function $\mathcal{Q}(ϴ,{ϴ}^0)$, or, equivalently, the set of coefficients $w_{gpk{k}^{\prime }}$ and ${\mathcal{Q}}_{gpk{k}^{\prime }}$. We accomplish this with the aid of an inward–outward recursion on the tree.

3.2.1.1. The Inward (γ) Recursion

Here we propose a variation on the well-known Felsenstein's pruning algorithm (26). Let us associate with each node t (except for the root) a vector ${\gamma }_i^{gpk{k}^{\prime }}\left(t\right)=\mathrm{Pr}(V_t\mid q_t^P=i,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)$. In words, ${\gamma }_i^{gpk{k}^{\prime }}$ (t) is the probability of observing the nodes V_t (which are a subset of the pattern ω_p) for a gene g, when the gain and loss rate variables are $r_k^{\eta }$ and $r_{k^{\prime }}^{\theta }$, respectively, and when the parent node of t is known to be in state i. By definition, this function is initialized at all leaves (t ∈ V₀) by

$$\gamma (t\in V_0)=\{\begin{array}{cc}\left(\begin{array}{c}\hfill 1-{\xi }_t(1-e^{-{\eta }_{gk}{\Delta }_t})\hfill \\ \hfill 1-(1-{\varphi }_t)e^{-{\theta }_{g{k}^{\prime }}{\Delta }_t}\hfill \end{array}\right)\hfill & q_t=0\hfill \\ \left(\begin{array}{c}\hfill {\xi }_t(1-e^{-{\eta }_{gk}{\Delta }_t})\hfill \\ \hfill (1-{\varphi }_t)e^{-{\theta }_{g{k}^{\prime }}{\Delta }_t}\hfill \end{array}\right)\hfill & q_t=1.\hfill \end{array}\phantom{\}}$$ [9]

Here, and in the derivations to follow, we omit the superscript from γ. For all internal nodes (except for the root), γ is computed using the recursion

$${\gamma }_i\left(t\right)=\sum _{j=0}^1{A}_{ij}(g,t){\tilde{\gamma }}_j\left(t\right),$$ [10]

where ${\tilde{\gamma }}_j\left(t\right)$ is defined as γ_j[L(t)]γ_j[R(t)] (see Note 2).

The γ-recursion allows for computing the likelihood of any observed pattern ω_p, given the values of the rate variables:

$$\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)=\mathrm{Pr}(V_0\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)=\mathrm{Pr}(V_0^L,V_0^R\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)==\sum _{i=0}^1\mathrm{Pr}(V_0^L,V_0^R,q_0=i\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)==\sum _{i=0}^1\mathrm{Pr}(q_0=i\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)\cdot \mathrm{Pr}(V_0^L\mid q_0=i,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)\cdot \mathrm{Pr}(V_0^R\mid V_0^L,q_0=i,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0).$$

Given q₀, $V_0^R$ is independent of $V_0^L$, and so

$$\mathrm{Pr}(V_0^R\mid V_0^L,q_0=i,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)=\mathrm{Pr}(V_0^R\mid q_0=i,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0),$$

and

$$\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)=\sum _{i=0}^1{\pi }_i{\tilde{\gamma }}_i\left(0\right).$$ [11]

This γ-recursion can be easily modified to incorporate missing data (see Note 3).

3.2.1.2. The Outward (α) Recursion

Once the γ-recursion is computed, we can use it to compute a second, complementary, recursion. To this end, let us associate with each node t (except for the root node) a matrix ${\alpha }_{ij}^{gpk{k}^{\prime }}\left(t\right)=\mathrm{Pr}(q_t=j,q_t^P=i\mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)$. It is beneficial to define for each node t (except for the root node) a vector ${\beta }_j^{gpk{k}^{\prime }}\left(t\right)={\sum }_{i=0}^1{\alpha }_{ij}^{gpk{k}^{\prime }}\left(t\right)=\mathrm{Pr}(q_t=j\mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)$. Upon the computation of α, β is readily computed too. Again, omitting the superscripts, α can be initialized from its definition on the two direct descendants of the root,

$$\alpha \left(\mathrm{D}\left(0\right)\right)=\frac{1}{\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)}\{\begin{array}{cc}\left(\begin{array}{cc}\hfill {\pi }_0{\gamma }_0\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right)A_{00}(g,\mathrm{D}\left(0\right))\hfill & \hfill 0\hfill \\ \hfill {\pi }_1{\gamma }_1\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right)A_{10}(g,\mathrm{D}\left(0\right))\hfill & \hfill 0\hfill \end{array}\right)\mathrm{D}\left(0\right)\in V_0,q_0^{\mathrm{D}}=0\hfill & \hfill \\ \left(\begin{array}{cc}\hfill 0\hfill & \hfill {\pi }_0{\gamma }_0\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right)A_{01}(g,\mathrm{D}\left(0\right))\hfill \\ \hfill 0\hfill & \hfill {\pi }_1{\gamma }_1\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right)A_{11}(g,\mathrm{D}\left(0\right))\hfill \end{array}\right)\mathrm{D}\left(0\right)\in V_0,q_0^{\mathrm{D}}=1\hfill & \hfill \\ \left(\begin{array}{cc}\hfill {\pi }_0{\gamma }_0\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right){\tilde{\gamma }}_0\left(\mathrm{D}\left(0\right)\right)A_{00}(g,\mathrm{D}\left(0\right))\hfill & \hfill {\pi }_0{\gamma }_0\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right){\tilde{\gamma }}_1\left(\mathrm{D}\left(0\right)\right)A_{01}\left(\mathrm{D}\left(0\right)\right)\hfill \\ \hfill {\pi }_1{\gamma }_1\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right){\tilde{\gamma }}_0\left(\mathrm{D}\left(0\right)\right)A_{10}\left(\mathrm{D}\left(0\right)\right)\hfill & \hfill {\pi }_1{\gamma }_1\left(\stackrel{‒}{\mathrm{D}}\left(0\right)\right){\tilde{\gamma }}_1\left(\mathrm{D}\left(0\right)\right)A_{11}\left(\mathrm{D}\left(0\right)\right)\hfill \end{array}\right)\hfill & \mathrm{D}\left(0\right)\notin V_0.\hfill \end{array}\phantom{\}}$$ [12]

Here, D(0) stands for any one of the direct descendants of the root, and D(0) is its sibling. For any other internal node, α is computed using the outward-recursion

$$\alpha \left(t\right)=\left(\begin{array}{cc}\hfill {\beta }_0\left(\mathrm{P}\left(t\right)\right){\tilde{\gamma }}_0\left(t\right)A_{00}(g,t)∕{\gamma }_0\left(t\right)\hfill & \hfill {\beta }_0\left(\mathrm{P}\left(t\right)\right){\tilde{\gamma }}_1\left(t\right)A_{01}(g,t)∕{\gamma }_0\left(t\right)\hfill \\ \hfill {\beta }_1\left(\mathrm{P}\left(t\right)\right){\tilde{\gamma }}_0\left(t\right)A_{10}(g,t)∕{\gamma }_1\left(t\right)\hfill & \hfill {\beta }_1\left(\mathrm{P}\left(t\right)\right){\tilde{\gamma }}_1\left(t\right)A_{11}(g,t)∕{\gamma }_1\left(t\right)\hfill \end{array}\right)$$ [13]

(see Note 4).

Finally, for each leaf that is not a descendant of the root,

$$\alpha \left(t\right)=\{\begin{array}{cc}\left(\begin{array}{cc}\hfill {\beta }_0\left(\mathrm{P}\left(t\right)\right)\hfill & \hfill 0\hfill \\ \hfill {\beta }_1\left(\mathrm{P}\left(t\right)\right)\hfill & \hfill 0\hfill \end{array}\right)\hfill & q_t=0\hfill \\ \left(\begin{array}{cc}\hfill 0\hfill & \hfill {\beta }_0\left(\mathrm{P}\left(t\right)\right)\hfill \\ \hfill 0\hfill & \hfill {\beta }_1\left(\mathrm{P}\left(t\right)\right)\hfill \end{array}\right)\hfill & q_t=1.\hfill \end{array}\phantom{\}}t\in V_0,\mathrm{P}\left(t\right)\ne 0$$ [14]

Again, this recursion can be straightforwardly modified when missing data are present (see Note 5).

These inward–outward recursions are the phylogenetic equivalent of the backward–forward recursions known from hidden Markov models, and other versions of it have already been developed (27, 28). The version that we developed here can be shown to be the realization of the junction tree algorithm (29) on rooted bifurcating trees (see Note 6).

3.2.1.3. Computing the Coefficients w_gpkk′

Here we show that the γ-recursion is sufficient to compute the coefficients w_gpkk′. From the definition, $w_{gpk{k}^{\prime }}=\mathrm{Pr}({\rho }_p^{\eta }=k,{\rho }_p^{\theta }=k^{\prime }\mid {\omega }_p,{\Xi }^0,{\Psi }_g^0,{\Lambda }^0)$. Using the Bayes formula Pr(x, y|z) = Pr(x, y, z)/Σ_x,y Pr(x, y, z), we can rewrite it as

$$\begin{array}{cc}\hfill w_{gpk{k}^{\prime }}& =\frac{\mathrm{Pr}({\rho }_p^{\eta },=k,{\rho }_p^{\theta }=k^{\prime },{\omega }_p\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)}{\sum _{h,h^{\prime }}\mathrm{Pr}({\rho }_p^{\eta }=h,{\rho }_p^{\theta }=h^{\prime },{\omega }_p\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)}=\hfill \\ \hfill & =\frac{\mathrm{Pr}({\rho }_p^{\eta }=k\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)\cdot \mathrm{Pr}({\rho }_p^{\theta }=k^{\prime }\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)\cdot \mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)}{\sum _{h,h^{\prime }}\mathrm{Pr}({\rho }_p^{\eta }=h\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)\cdot \mathrm{Pr}({\rho }_p^{\theta }=h^{\prime }\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)\cdot \mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gh{h}^{\prime }}^0)}.\hfill \end{array}$$

But $\mathrm{Pr}({\rho }_p^{\eta }=k\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)$ is just the current estimate of the probability of the gain rate variable to have the value $r_k^{\eta }$, namely ${\left(f_k^{\eta }\right)}^0$. Similarly, $\mathrm{Pr}({\rho }_p^{\theta }=k^{\prime }\mid {\Xi }^0,{\Psi }_g^0,{\Lambda }^0)$ is just ${\left(f_{k^{\prime }}^{\theta }\right)}^0$. Therefore, the expression for the coefficients $w_{gpk{k}^{\prime }}$ is reduced to

$$w_{gpk{k}^{\prime }}=\frac{{\left(f_k^{\eta }\right)}^0{\left(f_{k^{\prime }}^{\theta }\right)}^0\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)}{\sum _{h,h^{\prime }}{\left(f_h^{\eta }\right)}^0{\left(f_{h^{\prime }}^{\theta }\right)}^0\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gh{h}^{\prime }}^0)}.$$ [15]

The function $\mathrm{Pr}({\omega }_p\mid {\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)$ is the likelihood of observing pattern ω_p for gain and loss rate variables $r_k^{\eta }$ and $r_{k^{\prime }}^{\theta }$, respectively. This is readily computed upon completion of the γ-recursion, using Eq. [11].

3.2.1.4. Computing the Coefficients Q _gpkk'

Here we show that these coefficients require the α, β-recursion. By definition,

$${\mathcal{Q}}_{gpk{k}^{\prime }}=\sum _{\sigma }\mathrm{Pr}(\sigma \mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)\cdot [\text{log}{f}_k^{\eta }+\text{log}{f}_{k^{\prime }}^{\theta }+\text{log}\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})].$$

The probability $\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})$ is just the likelihood of a particular realization of the tree, thus from Eq. [1]

$$\text{log}\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})=\sum _{i=0}^1\delta (q_0,i)\cdot \text{log}{\pi }_i+\sum _{i,j=0}^1\sum _{t=1}^{N-1}\delta (q_t,j)\delta (q_t^P,i)\cdot \text{log}{A}_{ij}(g,t).$$ [16]

Here, δ(a, b) is the Kronecker delta function, which is 1 for a = b and 0 otherwise. Denote the expectation over $\mathrm{Pr}(\sigma \mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)$ by E_σ. Applying it to Eq. [16], we get

$$E_{\sigma }\left[\text{log}\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})\right]=\sum _{i=0}^1\text{log}{\pi }_i\cdot E_{\sigma }\left[\delta (q_0,i)\right]+\sum _{i,j=0}^1\sum _{t=1}^{N-1}\text{log}{A}_{ij}(g,t)\cdot E_{\sigma }\left[\delta (q_t,j)\delta (q_t^P,i)\right].$$

But $E_{\sigma }\left[\delta (q_0,i)\right]=\mathrm{Pr}(q_0=i\mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)={\beta }_i\left(0\right)$, and similarly $E_{\sigma }\left[\delta (q_t,j)\delta (q_t^P,i)\right]={\alpha }_{ij}\left(t\right)$. Hence, ${\mathcal{Q}}_{gpk{k}^{\prime }}$ is given by

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{gpk{k}^{\prime }}& =\sum _{\sigma }\mathrm{Pr}(\sigma \mid {\omega }_p,{\Xi }^0,{\Psi }_{gk{k}^{\prime }}^0)[\text{log}{f}_k^{\eta }+\text{log}{f}_{k^{\prime }}^{\theta }+\text{log}\mathrm{Pr}({\omega }_p,\sigma \mid \Xi ,{\Psi }_{gk{k}^{\prime }})]\hfill \\ \hfill & =\text{log}{f}_k^{\eta }+\text{log}{f}_{k^{\prime }}^{\theta }+\sum _{i=0}^1{\beta }_i\left(0\right)\text{log}{\pi }_i+\sum _{i,j=0}^1\sum _{t=1}^{N-1}{\alpha }_{ij}\left(t\right)\text{log}{A}_{ij}(g,t).\hfill \end{array}$$ [17]

3.2.2. The M-Step

Substituting Eq. [17] in Eq. [8], we obtain an explicit form of the function whose maximization guarantees stepping up-hill in the likelihood landscape,

$$\begin{array}{cc}\hfill \mathcal{Q}=& \sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}{n}_{gp}{w}_{gpk{k}^{\prime }}(\text{log}{f}_k^{\eta }+\text{log}{f}_{k^{\prime }}^{\theta })+\hfill \\ \hfill & +\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}{n}_{gp}{w}_{gpk{k}^{\prime }}[{\beta }_0^{gpk{k}^{\prime }}\left(0\right)\text{log}{\pi }_0+{\beta }_1^{gpk{k}^{\prime }}\left(0\right)\text{log}{\pi }_1]+\hfill \\ \hfill & +\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}\sum _{t=1}^{N-1}{n}_{gp}{w}_{gpk{k}^{\prime }}{\alpha }_{00}^{gpk{k}^{\prime }}\left(t\right)\text{log}[1-{\xi }_t(1-e^{-{\eta }_{gk}{\Delta }_t})]+\hfill \\ \hfill & +\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}\sum _{t=1}^{N-1}{n}_{gp}{w}_{gpk{k}^{\prime }}{\alpha }_{01}^{gpk{k}^{\prime }}\left(t\right)[\text{log}{\xi }_t+\text{log}(1-e^{-{\eta }_{gk}{\Delta }_t})]+\hfill \\ \hfill & +\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}\sum _{t=1}^{N-1}{n}_{gp}{w}_{gpk{k}^{\prime }}{\alpha }_{10}^{gpk{k}^{\prime }}\left(t\right)\text{log}[1-(1-{\varphi }_t)e^{-{\theta }_{g{k}^{\prime }}{\Delta }_t}]+\hfill \\ \hfill & +\sum _{g=1}^G\sum _{p=1}^{\Omega }\sum _{k=1}^{K_{\eta }}\sum _{k^{\prime }=1}^{K_{\theta }}\sum _{t=1}^{N-1}{n}_{gp}{w}_{gpk{k}^{\prime }}{\alpha }_{11}^{gpk{k}^{\prime }}\left(t\right)[\text{log}(1-{\varphi }_t)-{\theta }_{g{k}^{\prime }}{\Delta }_t].\hfill \end{array}$$ [18]

Actually, any increase in $\mathcal{Q}$ is sufficient to guarantee an increase in the likelihood, suggesting that a precise maximization of $\mathcal{Q}$ is not very important. Therefore, we speed computations by performing low-tolerance maximization with respect to each of the parameters individually. Except for the parameters λ_η and λ_θ, it is easy to differentiate $\mathcal{Q}$ twice with respect to any parameter. This lends itself into using simple zero-finding algorithms; we chose the Newton-Raphson algorithm (30). Maximizing $\mathcal{Q}$ with respect to the shape parameters λ_η and λ_θ is more involved, as $\mathcal{Q}$ depends on these parameters only through the discrete approximation of the rate variability distributions, Eq. [3] (see Note 7).